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From Celebration to 

Critical Investigation: 

Charting the Course of 
Scholarship in Applied Learning 


ROY SCHWARTZMAN 


University of North Carolina at Greensboro 


KELLY BOUAS HENRY 


Missouri Western State University 


Applied learning refers to learning experiences that take place outside traditional class- 
room settings. Examples include study away, service-learning, undergraduate research, 
and internship/practica/clinical experiences. As these pedagogies have increased in fre- 
quency over the past twenty years, the number of outlets devoted to the publication of 
scholarly work related to them has not kept pace. The Journal of Applied Learning in 
Higher Education (JALHE) attempts to fill that gap, providing an outlet for research and 
theory that critically examines applied learning’s impact and purpose using multiple meth- 
odological and disciplinary approaches. The initial volume of JALHE showcases work 
in this vein from a wide variety of disciplinary backgrounds and highlights areas ripe for 
future research. 


Now is the time, we conclude, to build bridges across the disciplines, 
and connect the campus to the larger world. Society itself has a great 
stake in how scholarship is defined. (Boyer, 1990, p. 77) 


Ernest Boyer’s provocative epigraph captures much of what the 
Journal of Applied Learning in Higher Education (JALHE) seeks to ac- 
complish. Applied learning figures prominently in national higher edu- 
cation reform efforts. In an era when few, if any, of higher education’s 
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long-held presumptions, such as the value of the liberal arts and of tenured 
professors, can be taken for granted, educational leaders have become 
more responsive to the demands of various stakeholders—businesses, 
governing boards, legislatures, parents, community organizations, local 
residents, regional accreditation agencies, and students. In particular, 
higher education institutions are learning to use scarce resources more 
efficiently to adapt to multiple learning styles and to produce measurable 
learning outcomes that meet the needs of industry and society. 

The reform of scholarly research charts a similar path. Boyer (1990) 
comments that “future scholars should be asked to think about the use- 
fulness of knowledge, to reflect on the social consequences of their 
work, and in so doing gain understanding of how their own study relates 
to the world beyond the campus” (p. 69). He goes on to praise fieldwork 
experiences in various disciplines “that involve students in clinical ex- 
perience and apprenticeships” (Boyer, 1990, p. 70). The evolution of 
scholarship clearly involves a deepening concern for the experiential as- 
pects of education. This essay will chart the course of JALHE by briefly 
tracing its historical and intellectual lineage, then embark on navigating 
it through the ongoing scholarly dialogues related to educational theory 
and practice. 


CONCEPTUAL TOPOGRAPHY OF APPLIED LEARNING 


Before proceeding, a definition of the central term in this journal’s 
title requires clarification. “Applied learning” refers more to a spirit or 
movement in education than to a definitively bounded subject matter. It 
designates the kinds of pedagogical principles and practices associated 
with engaged scholarship, communities of practice, civic engagement, 
experiential education, and critical pedagogy. Diverse as applied learn- 
ing may appear, all its manifestations share certain characteristics. Con- 
crete experience, “learning by doing,” lies at the core of applied learn- 
ing. This pedagogy represents active learning at its most literal level, the 
activity of putting intellectual principles into practice. 

Applied learning may be curricular or co-curricular, connected with 
coursework or a learning experience that occurs through other institu- 
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tional means (such as student service projects). These practices always 
have a central educational component that—as service-learning prac- 
titioners quickly observe—distinguishes them from volunteerism con- 
ducted solely for its intrinsic value. Although they extend beyond con- 
ventional classroom education, applied learning practices complement 
rather than replace other pedagogical methods. Applied learning typical- 
ly becomes manifest in higher education as one or more of the following 
kinds of pedagogical practices: study away (in an off-site environment, 
such as studying abroad or community-based learning), service-learn- 
ing, independent research, and internships/practica/clinical experiences. 
These practices always have a central educational component which dis- 
tinguishes them from out-of-class activities conducted solely for their 
intrinsic humanitarian value. 

The distinction between applied learning and more abstract theoreti- 
cal knowledge is articulated most crisply by philosopher Gilbert Ryle. 
According to Ryle (1949), intelligence melds two kinds of knowledge, 
which he labeled “knowing that” and “knowing how.” “Knowing that” 
encompasses theoretical understanding, the propositional knowledge 
typically reflected in mastery of facts and principles. “Knowing how” 
involves the demonstration of skill in performing a task. The integration 
of these two types of knowledge proves crucial in what counts as intel- 
ligence. Imagine someone learning how to play a game. The “knowing 
that” component deals with internalizing the rules to enable play. The 
“knowing how” aspect refers to the ability to execute moves in the game. 
Scholarship on applied learning investigates this relationship between 
intellectual understanding (comprehending the rules, knowing the sys- 
tem) and skillful practice (taking appropriate action). 

Theoretical knowledge without practical application creates the 
Ivory Tower intellectual incompetent to face the everyday challenges of 
life. In the film Defiance (2008), a Jewish refugee who cannot hammer a 
nail describes his vocation as “an intellectual,” which generates the puz- 
Zled reply from his comrade: “This is a job?” On the other hand, practical 
skill without theoretical understanding cannot generate desired results 
consistently. The swiftest runner cannot win the race without knowing 
which direction to run. Ryle (1949) suggests that blending theory with 
practice enables a learner to transcend mere training and move toward 
initiating the self-discipline that characterizes lifelong learning: “To be 
intelligent is not merely to satisfy criteria, but to apply them; to regulate 
one’s actions and not merely to be well-regulated” (p. 28). 

The concepts of applied learning often infused the educational lit- 
erature in discussions of vocational education. Researchers noted that a 
knowledge-based economy would require the flexibility to apply knowl- 
edge to a variety of tasks, and this versatility could arise only through put- 
ting theory into practice by engaging in concrete activities beyond the con- 
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fines of the traditional classroom (Kolde, 1991). Ongoing calls for greater 
relevance of higher education have helped to accelerate the pace of applied 
learning and extend it beyond job training. For most of society beyond aca- 
demia, the true test of knowledge lies in its connection to lived experience. 


RATIONALE: WHY THIS JOURNAL? WHY NOW? 


The development of any new scholarly journal responds to an intel- 
lectual need, the proverbial “gap in the literature” recognized in every 
thesis or dissertation. The need for this journal is both profound and 
persistent. Applied learning programs have proliferated far more rapidly 
than the scholarly tools to examine the merits of their design and the 
measures of their effectiveness. While the popularity of applied learning 
generates celebration that the Ivory Tower is becoming part of the sur- 
rounding neighborhood, it also triggers consternation. As mushrooming 
numbers of individual case studies accumulate, the speed of implement- 
ing various applied learning practices has outpaced the ability to deter- 
mine systematically what works best, when, and why (Densmore, 2000; 
Eyler, 2002). 

The chronological history of this journal begins with the Conference 
on Applied Learning in Higher Education (CALHE), developed and 
hosted by Missouri Western State University in St. Joseph. The confer- 
ence was born out of Western’s statewide mission as Missouri’s “applied 
learning” institution, a designation that became official in 2005. Like 
many universities, Western already had institutionalized many forms of 
applied learning experiences. Beyond simply practicing applied learn- 
ing, the university sought to implement its mission by providing a venue 
to promote best practices of applied learning that could serve the state, 
region, and nation. The conference emerged as a way to meld the often 
abstract realm of institutional mission statements with the often under- 
theorized and under-analyzed practice of applied learning. CALHE of- 
fered to bring state-of-the-art training, research, and analysis through 
the top experts in various areas of applied learning—thereby putting the 
university’s mission into practice through scholarship and intellectual 
dialogue. 

Shortly after the first CALHE in 2006, we began to see that a related 
way to support the university’s mission and to deepen the conversations 
regarding applied learning was to develop a peer-reviewed outlet that 
faculty and practitioners could turn to as a way to engage in scholarly 
discussions related to applied learning. The conference was becoming a 
provocative avenue for sharing ideas about applied learning across vari- 
ous disciplines. Unfortunately, it was limited by the ephemeral nature 
of such events. Energetic exchanges of ideas occur, stimulating further 
reflection, and then those exciting conversations gradually fade after 
the conference concludes. The narcotic everyday routine of paperwork 
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and other mundane tasks quells the momentum built at the meeting. The 
journal offers an ongoing, permanent resource for restoring and sustain- 
ing vibrant intellectual discourse. 

The journal also addresses an issue that has emerged on many college 
and university campuses. CALHE emerged at a time when higher educa- 
tion was (and still is) embroiled in one of many dilemmas that pit tra- 
ditional academic perception against current academic practice. On one 
hand, the realities of academic life at all levels place increasing demands 
on faculty to serve and teach. The chronic shortage of personnel willing 
to engage in governance and leadership means that a broader range of 
these administrative duties filters into the everyday duties of faculty and 
staff. Increasing teaching and service loads driven by demands for ef- 
ficiency translate to more students and more tasks with fewer resources. 
Time for discipline-based research may erode in the face of these de- 
mands, especially in a social climate where only classroom instruction 
is conceived as “genuine” academic labor. On the other hand, promo- 
tion and tenure committees still tend to prioritize scholarship, especially 
at research-oriented institutions. How can these competing demands 
be reconciled? 

Applied learning often falls through the cracks of faculty evalua- 
tion and reward systems in higher education. If understood solely as 
a pedagogical practice, it remains segregated in the “teaching” realm 
of evaluation. The conference and the journal treat the boundaries be- 
tween teaching, research, and service as permeable. Too often, espe- 
cially at research-oriented universities, teaching and service occupy dis- 
tinctly lower levels in the evaluative hierarchy. The Journal of Applied 
Learning in Higher Education is dedicated to demonstrating that ped- 
agogical practice—much of which involves service—and rigorous 
research are not only compatible, but symbiotic. The nature of the 
journal reflects the mutual fertilization among the four types of schol- 
arship Boyer (1990) articulates: scholarship of discovery, scholarship 
of integration, scholarship of teaching, and scholarship of application. 
Within the pages of this journal, the practices and principles of different 
disciplines are integrated by their shared goal: to probe the theoretical 
grounds, best practices, and implications of applied learning in all its 
forms. This journal also provides a scholarly forum for conducting the 
scholarship of engagement (Boyer, 1996), the examination of pedagogi- 
cal practices that implement the traditional institutional mission to serve 
its community and constituencies. This scholarly approach to service- 
related activities reunites the public service activities of higher education 
with the critical analysis that constitutes the core of academic research. 

Some excellent peer-reviewed journals currently address different 
aspects of applied learning, but few scholarly avenues have provided a 
single forum to engage multiple disciplines in discussions of all aspects 
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of applied learning. By providing such a venue, this journal broadens the 
reach of scholarship beyond disciplinary restrictions (thereby fostering 
integration) and adds depth to scholarship of teaching by covering mul- 
tiple pedagogical practices (not only undergraduate research, service- 
learning, etc.). In many ways, the movement of engaging students in 
experiential learning has grown up in silo form, with different forms of 
applied learning having their own organizations and emphasizing dif- 
ferences between disciplines and practices rather than seeking shared 
concerns with the nature and practice of applied learning per se. 

The different forms of applied learning can inform each other and 
need not remain segregated in separate journals. For example, faculty 
who have for many years engaged in undergraduate research have used 
critical reflection to further student learning (though often such work 
was couched in analysis of methodological design and statistical infer- 
ence). Practitioners of service-learning who wish to go beyond reflection 
based on sharing reactions to more analytical reflective practice could 
probably learn a great deal from the work of undergraduate research di- 
rectors and practitioners on how to integrate that sort of critical reflec- 
tion into a service-learning project. In addition, undergraduate research 
directors might be pushed by faculty who think a lot about finding good 
internship sites to consider ways to integrate undergraduate research into 
internship experience that relates the research to potential employment. 

In its fetal stages, CALHE began as an internal, single-institution 
poster session with fewer than ten poster presentations. In 2009, the 
conference had more than 200 registrants from 17 states and Austra- 
lia. Through JALHE, those who are committed to applied learning, and 
the scholarship of teaching and application, will have a way to docu- 
ment and disseminate their work. The journal’s promotion of deeper 
discourse between the different forms of applied learning can build a 
foundation for understanding and furthering best practices in all forms of 
experiential education. 


THEORETICAL MILIEU OF APPLIED 
LEARNING RESEARCH 


Many years of reviewing conference papers, journal manuscripts, 
and grant proposals reveal a narrative structure that has become too 
common in the scholarship on applied learning. The structure constitutes 
what could be called the cheerleading model of self-advocacy. Too many 
submittals to conferences, journals, and grant review boards use a posi- 
tive experience with a particular applied learning experience to make a 
generic argument in favor of applied learning per se. These self-congrat- 
ulatory narratives of unqualified success invite readers to worship at the 
altar of applied learning. The scholarship on applied learning, however, 
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must move from love fest to knowledge quest. A productive first step in 
this direction is to recognize and discuss the pitfalls of projects, the lim- 
its of experiential learning, and more nuanced ways to evaluate quality 
of outcomes. For example, the problems associated with teaching about 
the Holocaust via simulated starvation diets and mock persecutions has 
led the Anti-Defamation League (2006) to issue a statement condemn- 
ing simulation-based Holocaust education. Research on best practices 
should offer insight about what to avoid as well as what to emulate. 
Rather than offer a one-shot case study with the narrative pattern “Lo, 
we did it, and behold: it was good,” research must probe the rationale 
for the results it reports. How does a particular project fit within the 
broader intellectual conversation regarding how to structure, administer, 
and assess applied learning? Another way to pose this question would 
be to inquire, along the lines of Fink (2003), about how a specific study 
is driven by and expands upon the relevant pedagogical and disciplinary 
theories. 

Part of this larger conversation concerns the role applied learning 
can play in the discourse regarding higher education. Several examples 
illustrate the intriguing issues studies of applied learning might confront. 
One issue involves equipping students to become more independent, 
lifelong learners. Early exposure to a directed research experience might 
enable students who are conditioned to become received knowers—pas- 
sively and obediently responding to authority but not initiating ideas—to 
acquire greater ownership over their ideas. Guided research activities 
could recondition students to become more independent thinkers by pri- 
oritizing invention over retention. Rather than implement a master plan 
of “please the teacher” by repeating whatever the authority figure wants 
to hear, research could guide the capacity for creative risk-taking, a key 
factor in critical thinking and problem solving. Future studies might 
investigate the connection between such research experiences and 
innovative student achievements beyond the classroom. 

Apprehension about the competing forces of efficiency and effective- 
ness looms over the higher education landscape. The narrow constraints 
of commodification have positioned students as consumers to be placat- 
ed, but as cheaply and quickly as possible (McMillan & Cheney, 1996; 
Schwartzman, 1995; Schwartzman & Phelps, 2002). This discourse of 
efficiency clashes with the competing pressures of demonstrable effec- 
tiveness. The result: paradoxical mandates to increase class sizes but 
also increase “customer service” to each student, or broaden access to 
higher education but improve standardized test scores and accelerate 
graduation rates. The list could continue indefinitely. Applied learning, 
however, offers pedagogical practices that have navigated the contrast- 
ing demands of higher education and external constituencies. For exam- 
ple, what lessons from sustainable campus-community partnerships in 
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service-learning projects might generalize to campus-corporate adminis- 
tration of internships? How does applied learning demonstrate the practi- 
cal relevance of educational experience (a sine qua non of commodifica- 
tion) while also improving field-specific intellectual expertise (the essence 
of academe)? 


METHODOLOGY MATTERS 

Thus far, we have discussed the “‘what,” “when,” and “why” of schol- 
arship on applied learning by tracing its theoretical parameters, intellec- 
tual history, and social justification. Attention now turns to the “how,” 
which constitutes the modes of conducting research relating to applied 
learning. Much scholarship on applied learning relies on self-reports, 
especially from student participants, as a major source for evidence of 
learning outcomes and social effects. Such self-reports, especially when 
used as the sole data points, raise significant methodological concerns. 
These challenges should generate further scholarly reflection on how 
self-reports are used and should stimulate researchers to employ multiple 
methods that can produce more diverse documentation of applied learn- 
ing experiences. The following concerns should urge scholars to employ 
a variety of evaluative measures when seeking to determine the value of 
applied learning. 

Eyler (2002) notes that self-reports from student participants do not 
constitute sufficient evidence of successful educational outcomes or so- 
cial impact. She raises the fundamental issue of validity: anecdotal self- 
reports cannot consistently or systematically link experiential learning 
inputs (e.g., program design, student demographics, nature of the learn- 
ing experience) with personal or societal benefits. Schwartzman (2002) 
details some of the methodological challenges when he expresses con- 
cerns over the heavy reliance on self-reports in the service-learning lit- 
erature. First, many survey instruments measure attitudes regarding the 
applied learning experience. It becomes difficult to correlate attitudinal 
tendencies with behavioral outcomes, especially when the desired out- 
comes include long-range or ongoing behavioral changes. For example, 
studies may attempt to show a project increased civic engagement by 
reporting a significant increase in likelihood to participate in activities 
associated with responsible citizenship, such as voting. Yet, how many 
of these studies actually track whether participants eventually engage in 
the desired activities? More studies of observable effects through direct 
observation and longitudinal data collection can document not only the 
immediate success but the long-term staying power of applied learning. 

Excessive reliance on self-reports also invites systematic bias. 
Gelmon (2003) observes that community partners are likely to overem- 
phasize positive service-learning experiences and underplay any draw- 
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backs, because they fear that any negative feedback might jeopardize 
future supplies of labor. This concern could extend to other applied 
learning activities. Host universities might be reluctant to report negative 
experiences with study abroad students as they could endanger recipro- 
cal agreements with institutions that sent the students. Site supervisors of 
interns might offer overly positive performance appraisals to protect the 
continued infusion of low-cost labor. 

Self-reports from students may suffer from distortion on several lev- 
els, including social desirability bias, self-justification, and reciprocity. 
Each of these difficulties deserves further reflection. Discussing research 
on service-learning, Pritchard (2001) identifies social desirability and 
self-justification as factors that might taint results: “When surveys are 
used to evaluate the success of an effort, particularly at its completion, 
the respondents usually know what answers the evaluators want and are 
also predisposed to rationalizing their own investment of time and effort” 
(p. 24). Most applied learning projects include a rather explicit state- 
ment of desired outcomes. Even without a pre-established outcome for 
the specific project, the social and educational goals tend to be transpar- 
ent: study abroad increases cultural awareness, undergraduate research 
enables acquisition of discipline-specific knowledge, service-learning 
aids clients of community organizations, internships develop job-related 
skills. End-of-project surveys also invite false or exaggerated positives 
as ways to avoid cognitive dissonance that might arise from admitting 
failure to achieve the project’s objectives. Finally, reciprocity introduces 
potential distortions if respondents feel morally obligated to “return the 
favor” of a stakeholder by skewing evaluations toward the positive side. 
Even with properly designed questionnaires, a study abroad student may 
rank a host university’s academic programs as outstanding based on the 
institution’s hospitality rather than its academic quality. In fact, many as- 
sessments of applied learning experiences may require substantial recon- 
sideration to improve their validity. Minimally, the tendency to generate 
“false positives” should raise concerns about reporting self-evaluations 
absent some type of comparative scales with control populations that 
establish evaluative norms (Darby, 2008). 

Another methodological challenge lies in distinguishing satisfaction 
from learning outcomes and social impact. Too often, success of applied 
learning initiatives tends to be judged by measures of positive affect. 
Put more directly, popularity presumably proves success. Especially in 
times of economic constraint, applied learning must pay more tangible 
dividends than spreading happiness. Neil Postman (1984) has offered 
ongoing cautions about confusing entertainment with education, noting 
that highly engaged students may not have learned much despite their 
expressing deep satisfaction with a learning experience. 
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In fact, enthusiastic appreciation of applied learning needs to be 
tempered by carefully determining which measures would suffice to 
support the positive impact of an applied learning experience. The sup- 
port invoked here refers to what would count as evidence for various 
constituencies, including legislators and funding agencies. Frankly, few 
people outside the walls of academia find the results of student feedback 
particularly compelling because they recognize the subjectivity of even 
the most statistically saturated evaluations. One pedagogical strength 
of applied learning is that it offers many other types of measures from 
multiple stakeholders that can document outcomes. For example, a ser- 
vice-learning project could document the number of clientele a com- 
munity organization served prior to the project compared to the number 
the project reached. This kind of documentation provides demonstrable 
evidence that a project extended the capacity of a service agency, an 
especially powerful statement if the agency already has been recognized 
as vital to the community. 

When several indicators demonstrate similar outcomes, the conclu- 
sion becomes much stronger. Applied learning experiences are amenable 
to many assessment measures in addition to (or instead of) subjective 
self-reports from students. Several academic fields have developed quite 
refined measures of learning outcomes, and these field-specific mea- 
sures could be administered to compare students who undergo applied 
learning with their counterparts who participate in traditional classroom 
instruction. External constituencies, such as community partners for ser- 
vice-learning or professional practitioners for field experiences, could 
document how student involvement affected organizational practices or 
administer independent assessments (such as knowledge of field manu- 
als) to determine levels of practical knowledge. Interns might receive 
performance appraisals from multiple evaluators rather than only from 
a site supervisor who feels compelled to maintain a positive relationship 
with the academic institution. A long-term assessment of any participant 
in applied learning could consist of making that student a mentor for sub- 
sequent students, who would then evaluate the quality of the preparation 
they received for their educational experience. 

Research on applied learning can catapult beyond the “testimonial of 
success” mode by going beyond a one-dimensional pre-test/post-test de- 
sign. This methodology measures student opinions or learning outcomes 
before and after an applied learning experience and attributes improve- 
ments to the project. Frequently these designs fail to incorporate controls 
or benchmarks that would enable the researcher to isolate the “value 
added” uniquely from the project itself. For example, would similar out- 
comes have resulted from less expensive, less labor-intensive, less risky 
instructional techniques? How do the experiential project’s outcomes 
compare with the same material taught in a non-experiential format? 
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Does the study control for the instructor or supervisor of the project, 
or are outcomes the artifact of the administrator rather than stemming 
from the project itself? Which demographic factors are controlled? For 
example, does a study discussing the impact of a study abroad program 
include comparisons or controls across different nations? Could the 
demonstrated impact of a study abroad program result more from the 
nature of a particular national culture or university system than from the 
international experience? 

The applied learning literature also could benefit from more vigorous 
longitudinal studies, especially those that track long-term behavioral pat- 
terns attributable to applied learning experiences. Some excellent work 
has been done on tracking specific programs, such as service-learning 
partnerships, over time (e.g., Keen & Hall, 2009; Kiely, 2005). Still, 
few findings have accumulated that document the extent to which stu- 
dents independently embark on activities consistent with the objectives 
of their applied learning experiences. In short, are students applying the 
lessons of their applied learning? The research questions in this area 
seem deceptively obvious, yet they have paramount significance for 
the long-term justification of applied learning practices. Consider only 
a few examples. Do students who study abroad initiate or participate 
more frequently in organizations that further intercultural awareness and 
collaboration? To what extent do international experiences reduce eth- 
nocentric attitudes and behaviors? How are the membership numbers 
of international student groups correlated to the rates of study abroad 
experiences? Do study abroad participants exhibit distinctive patterns 
in their career choices that leverage the value of their international ex- 
perience? Have field experiences been correlated with better qualified 
applicants for positions in those fields? How do in-field placement rates 
of student researchers compare with those of students who did not en- 
gage in undergraduate research? How has the undergraduate research 
experience equipped alumni to (a) conduct research outside their field, 
or (b) embark on professional duties that do not involve research in the 
academic sense? Closer tracking of student participants as well as other 
constituencies over an extended period of time would begin to answer 
these questions and many more that linger. 

Research on applied learning could reap substantial benefits from 
mining the rich but often untapped data from voices rarely heard in 
scholarship that plucks the low-hanging fruit of student self-reports. 
More extensive explorations of input from constituencies such as applied 
learning program administrators (e.g., service-learning, study away, and 
internship coordinators), community members, site supervisors, clien- 
tele served in community-based projects, or students and faculty in host 
universities abroad would broaden the conversation beyond faculty re- 
porting their students’ opinions to other faculty. The impact of applied 
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learning research also would intensify if it embraced public policy issues 
more directly. For example, how do legislators and funding agencies 
define successful applied learning? How do their criteria and percep- 
tions square with those of academics? What might account for or correct 
misalignments? How could academic expectations for applied learning 
interface better with desired political and social outcomes? 


IN THIS ISSUE...AND BEYOND 


The contents of this issue span a broad scope of applied learning, 
illustrating the range of pedagogical practices that can generate produc- 
tive intellectual discussion. The articles begin with an analysis of the 
role of reflection in applied learning and how to develop reflection tools 
that can facilitate learning while accurately documenting the nature of 
the learning experience. The scholarship in the rest of the issue covers 
a wide array of concerns: an alternative certification program centered 
on experiential components, a field experience that connects coursework 
with professional practice, the impact a rubric to assess student learning 
can have on student motivation in an experiential setting, and a service- 
learning project that enables teacher candidates to develop their Spanish 
skills while teaching basic English. Together, these articles provide a 
sample of the range of scholarship that applied learning can generate. 
Individually, each article raises issues and questions that can stimulate 
ongoing research. 

“Generating, Deepening, and Documenting Learning: The Power of 
Critical Reflection in Applied Learning,” by Sarah L. Ash and Patti H. 
Clayton, addresses the importance of the careful and purposeful use of 
reflection to motivate and measure student learning outcomes. Ash and 
Clayton (2009) note that the term “reflection” is somewhat problematic 
in that it suggests a reactive, emotional analysis to the events that take 
place in an applied learning situation rather than a critical analysis of 
those events. Thus they use the term “critical reflection” to connote this 
more analytical process that has been linked so closely to student learn- 
ing (Eyler & Giles, 1999). Henry and Kempf (2005) found that faculty 
may use critical reflection far less than they actually prefer. Ash and 
Clayton call for faculty who supervise applied learning to intentionally 
design critical reflection around learning goals. The piece provides an 
enormous resource to faculty who too often rely on an end-of-term re- 
flection paper (which rarely gets feedback to students in a timely fash- 
ion) or journaling that degenerates into sharing observations rather than 
analyzing those observations in light of learning goals for the course. 

In addition to providing a powerful pedagogical tool, Ash and 
Clayton’s work inspires us to think in terms of how student assessment 
might not just feed into a gradebook, but into a scholarly analysis of 
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how well our instructional design worked in terms of facilitating learn- 
ing goals. Student learning assessed using their Describe-Examine- 
Articulate Learning (DEAL) model informs a faculty member’s assess- 
ment of his or her own work. For instructors who want to publish their 
work as scholarship of teaching, the DEAL model can generate compel- 
ling evidence of the effectiveness of a given applied learning experience 
beyond Likert-type evaluations that may measure student satisfaction as 
much as learning (Greenwald & Gillmore, 1997; Snare, 2000). The abil- 
ity to generate such an argument is increasingly tied to funding support 
for activity in higher education. Particularly in tight financial times, this 
sort of assessment tool is critical in demonstrating the impact of applied 
learning and its value to external constituents. 

Finally, Ash and Clayton’s approach to applied learning lays out a 
research process by which faculty develop, in a sense, hypotheses about 
what strategies will positively impact student learning and test those hy- 
potheses using student critical reflection products. This approach allows 
faculty to move beyond scholarly teaching toward the critical evaluation 
and testing of one’s own strategies that can be submitted for peer review. 
Indeed, work that follows this line of analysis would be most welcome in 
future volumes of the Journal of Applied Learning in Higher Education. 

“Examining the Development of the Victorian Certificate of Applied 
Learning and Its Implications for Schools and Teacher Education in 
Australia,” by Damian Blake and David Gallagher, provides an out- 
standing example of what can be accomplished when applied learning 
is implemented system-wide. It also highlights the implications such an 
adoption can have on teacher training programs, which implies that there 
may be unanticipated impact in other disciplines that seek to integrate 
applied learning across the curriculum. 

The Victorian Certificate of Applied Learning (VCAL) was moti- 
vated by a desire to increase graduation rates from secondary schools in 
the province. The baseline completion rate of senior high school prior to 
the development of an alternate educational path based in applied learn- 
ing pedagogy was 80%, with the goal that the VCAL might raise that to 
90%. Achieving that goal would add more than 60,000 individuals to 
the workforce, significantly boosting the regional economy. But why the 
VCAL, and not some other alternative route to completion, such as the 
GED program in the United States? Because the VCAL addresses the 
increased diversity that folds into an educational system trying to retain 
students who would otherwise leave school early. The VCAL provides a 
means for these students to earn legitimate academic credit for learning 
that occurs in non-traditional situations. The process involves creating 
a dialogue with students about the curriculum through which curricular 
content is negotiated and ultimately assessed. 
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The assessment component to the VCAL is critical to the integrity 
of such programs. It is part of the reason that the VCAL had significant 
implications for teacher training. For teachers who are trained in tradi- 
tional assessment strategies, it can be difficult to imagine how a real- 
world experience could translate to assessment focused on test scores. 
The universities in the province educating future teachers had to prepare 
these individuals to handle a wider array of assessment strategies, as well 
as to deliver a wider array of pedagogical strategies. Given the difficulty 
in recruiting and retaining high-quality secondary school teachers in the 
United States, making the process broader and more rigorous is not a 
trivial problem. The key to doing so successfully may lie in the ability to 
demonstrate such a program’s economic impact. The VCAL has moved 
Victoria significantly toward the goal of 90% completion rates. In addi- 
tion to documenting completion rates, it will be important in future years 
to document the impact of that improved completion rate in terms of 
increased workforce and related productivity gains. 

In fact, this type of accountability is not only relevant to the VCAL, 
but may be a useful strategy for others doing research on applied 
learning. Applied learning is inordinately inefficient compared to tradi- 
tional lecture-based counterparts in the academy. In lean times, univer- 
sity administrators may question the value of continued support for such 
pedagogical strategies if the faculty who practice them have not care- 
fully documented the benefits of such investments for the constituents of 
the university. Future research in a variety of areas of applied learning 
will benefit from scholars who establish an economic value to the ex- 
pensive process of applied learning, as well as the impact on graduation 
rates and student learning. 

“Will They Recognize My Lecture in the Field? The Juvenile 

Corrections Critical Assessment Tour Applied Learning Experience,” 
by Greg Lindsteadt and Regina Williams-Decker, raises several impor- 
tant issues for applied learning. The authors note the stimulus for applied 
learning as pedagogical, but also as a step in rehabilitating the reputation 
of criminal justice as an academic field. This kind of reflection proves 
especially timely during economic belt-tightening, when academic pro- 
grams must appeal to (largely non-academic) policymakers by demon- 
strating practical relevance. Lindsteadt and Williams-Decker examine 
the ways a deep field experience within juvenile justice facilities can 
show the applicability of course content to the actual practice of juvenile 
justice. Writ large, this essay poses the question of how well class- and 
text-based theory aligns with the practices within the criminal justice 
system. 

The Juvenile Corrections Critical Assessment Tour (JCCAT) dis- 
cussed in the article invites reflection that extends far beyond the dy- 
namics of the specific applied learning experience, ambitious as it was. 


Schwartzman, Henry / SCHOLARSHIP ON APPLIED LEARNING — 17 


Usually the practical relevance of course content is judged unidirection- 
ally, by measuring correspondence of class experience with activities in 
the profession. Future research might invert the question these authors 
ask, querying whether practitioners will recognize future juvenile justice 
workers when they interact with the students. Applied learning experi- 
ences usually occur at the intersection of several realms of participants: 
students, academics, field workers, and clients in the field. It might 
prove rewarding to gather data not only on whether students recognize 
course content in the field, but also whether the personnel at juvenile 
justice facilities view the course content as an accurate depiction of the 
justice system. 

As for learning outcomes, the JCCAT opens the door to multiple 
measures of learning. The article offers intriguing glimpses of possible 
systemic biases in the actual practice of juvenile justice, especially in the 
areas of racial, gender, and class equity. The course as well as the project 
itself might in future iterations dig deeper into the roots of these dispari- 
ties—perhaps uncovering lingering, unresolved tensions between reha- 
bilitative models of justice and punitive practices that may contribute to 
recidivism more than reformation. On a broader level, the JCCAT expe- 
rience could confront the convergences and divergences between social 
justice and criminal justice (Rawls, 1971). Direct engagement with these 
systemic issues might equip students not simply to become future work- 
ers within the justice system, but rather to empower them as agents to 
rectify the discrepancies they observe between theories of justice and the 
ways justice is meted out to juveniles. 

“Student Motivation and Assessment of Applied Skills in an Equine 
Studies Program,” by K. I. Tumlin, R. Linares, and M. W. Schilling de- 
scribes the impact of using a rubric—and providing it to students prior 
to testing—to assess hands-on, psychomotor skills such as showmanship 
in an equine studies program. Theoretically, providing students with the 
rubric ahead of the assessment should clarify the grading standards and 
improve student performance on the applied tasks. In fact, the authors 
of this study report the counterintuitive finding that students who were 
given a rubric beforehand actually performed worse on the assessment 
of their applied skills than students who had not seen the rubric at all. 

Rubrics are standard operating procedures in much of the education- 
al domain, in part because they standardize assessment of skill and in 
part because they make the evaluation standards transparent to students. 
Indeed, McTighe and O’Connor (2005) argue that providing the rubric 
to students is a necessary (though not sufficient) condition for support- 
ing student learning. The finding that in an applied learning setting the 
rubric resulted in poorer student learning outcomes is intriguing. Future 
research might focus on establishing whether or not this result is repli- 
cable in other disciplines that use applied learning, and if the result is 
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reliable, the reason behind it. The authors of this study report that the 
students in this program are largely more goal-oriented than process- 
oriented. Could it be that providing students who are already goal- (i.e., 
grade?) oriented pushes them to achieve the minimum rather than the 
maximum possible? This is exactly what happened for the students in 
this study—the average performance was lower overall and the failure 
rate was lower when the rubric was present but the “superior” perfor- 
mance category was much more likely to be achieved when the rubric 
was not provided. 

This result invites the question of what impact a rubric has in an 
applied learning setting. In some ways, providing a rubric “frames” the 
situation for students a priori. Part of the point for those of us who 
encourage students to participate in applied learning activities such as 
practica in a discipline is to expose students to learning situations that 
are vague and undefined, requiring the student to frame the problems 
they encounter on-site for themselves. The ability to effectively frame 
a situation so that one can apply the appropriate academic content to 
generate solutions and productive work is critical to applied learning. 
Indeed, Eyler and Giles (1999) argue that “application” is one of the vital 
components of the learning process in service-learning settings. Does 
providing a rubric to students in an applied learning situation perhaps 
defeat part of the purpose of applied learning by effectively narrowing 
students’ attention to a limited set of features or possible solutions? The 
results reported by Tumlin, Linares, and Schilling invite future scholars 
to investigate these and other possibilities. 

“Demographic Tipping Point: A Discussion of Cultural Broker- 
ing with English Language Learners as Service-Learning for Teacher 
Candidates and Educators” by Wendy McCarty, Rosemary Cervantes, 
and Geraldine Stirtz details the experience of a service-learning project 
that illustrates the mutual impact such experiences can have on students 
and community members. The project implements “cultural broker- 
ing,” defined as intercultural partnerships initiated to instigate positive 
social change. The brokering in this case involves teacher candidates 
helping native Spanish speakers learn basic English. The language learn- 
ing moved in two directions, typifying the brokering relationship. The 
English language learners (ELLs) developed their linguistic skills to 
improve their social mobility, while the teacher candidates acquired 
more Spanish language skills through conversing with the learners and 
their families. 

The cultural brokering experience offers intriguing possibilities 
regarding how service-learning might combine with intercultural 
activities to enhance linguistic and intercultural competence. Spe- 
cifically, standardized pre-tests and post-tests of language skills could 
gauge the degree to which the teacher candidates and the ELLs learned 
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each other’s language. As for intercultural sensitivity, prior research on 
service-learning has noted the potential for intercultural experiences to 
trigger boomerang effects if the unfamiliar culture is cast in particular 
kinds of social roles (Erickson & O’Connor, 2000). Placing certain pop- 
ulations, such as non-native English speakers, consistently in the role of 
needing assistance might actually reinforce ethnocentric views of Anglo- 
American hegemony. Inventories of intercultural tolerance or measures 
of prejudice could be administered to determine how cultural brokering 
affects the cross-cultural attitudes of participants. 

As the authors note, participant feedback deserves supplementation 
with other kinds of empirical and interpretive analysis. One interesting 
direction for further research would be to employ different pedagogical 
techniques to teach the ELL students. The project could collect important 
data on the most effective ways to enable non-native English speakers 
to acquire English language skills. Do the same techniques for foreign 
language instruction of native English speakers work as well for stu- 
dents from other linguistic backgrounds? Which pedagogical techniques 
prove most effective with particular demographics of language learners? 
McCarty, Cervantes, and Stirtz open the door for future research to begin 
to mine a variety of data sources for answers to these questions. 


INVITATION TO FURTHER INVESTIGATION 


The finale of this article consists of suggestions for future research 
directions rather than a definitive conclusion that brings inquiry to a 
neat—and necessarily premature—conclusion. Scholarship on applied 
learning needs to move beyond self-advocacy. The justifications for 
applied learning have been made convincingly. These arguments require 
more nuanced examination to avoid categorical endorsement of applied 
learning regardless of its method or context of implementation. We sug- 
gest several modes of conducting scholarship on applied learning that 
go beyond self-justifications of particular applied learning experiences. 

Many fruitful paths for scholarship on applied learning await ex- 
ploration. Thus far, little attention has focused on the interface between 
applied learning and emergent educational technologies. Applied 
learning techniques have been discussed as low-tech ways to intensify 
intellectual experiences, especially when compared to hands-off, deper- 
sonalized methods that operate via economies of scale (Schwartzman, 
2001). Minimal attention, however, has been devoted to the role technol- 
ogy might play within applied learning. For example, what implications 
do computerized simulations have for field experiences and practica? 
As budgetary belts tighten, might virtual experience supplement, enrich, 
or displace what counts as experience in experiential learning? Rather 
than demonize new technological tools, future research could tap into the 


20 Journal of Applied Learning in Higher Education / Fall 2009 


instructional technology literature to investigate how technological tools 
and applied learning could prove mutually beneficial. How could inex- 
pensive means for touring sites and conducting long-distance interviews 
enhance preparation for study away? In what ways might collaborative 
and social networking tools from wikis to Twitter and beyond increase 
the sustainability of learning beyond the duration of a term spent study- 
ing abroad or at an internship site? Could virtual realities such as Second 
Life and similar realms intensify preparation for firsthand applied learn- 
ing experiences? Overall, how might applied learning leverage the pow- 
er of technology to (a) improve student readiness for applied learning, 
(b) enrich the applied learning experience, (c) extend the sustainability 
of applied learning beyond a project’s termination at a given locale, (d) 
improve methods of assessing learning outcomes, (e) reduce or maintain 
costs without sacrificing quality? 

In an important essay surveying the theoretical territory of the com- 
munication studies field, Craig (1999) identifies socio-cultural and criti- 
cal approaches as two classifications of communication theory. These 
perspectives transcend communication studies and could generate re- 
search that would enrich the study of applied learning. 

Socio-cultural scholarship on applied learning asks questions that 
explore the interaction between applied learning and its social environ- 
ment. Internships offer an excellent example of opportunities for socio- 
cultural explorations. Internships tend to be discussed from two per- 
spectives: the student’s pedagogical/professional development and the 
sponsoring organization’s labor needs. Less attention has been devoted 
to issues related to internships and organizational culture. One such issue 
involves the intern’s place in organizational hierarchies. If an internship 
serves as a training ground for executive positions, a hope many student 
interns harbor, then how do rank and file employees view the intern who 
serves a brief internship and then leapfrogs over employees with greater 
seniority to a position that outranks them? How does the rapid rise of an 
intern through the organizational ranks comport with an organizational 
culture that touts advancement through “paying your dues,” and “tak- 
ing care of our own’? If interns are analogous to apprentices, then how 
does an apprenticeship through an educational institution’s internship 
program compare to an apprenticeship of on-the-job training without an 
academic component? 

Internships also provide an opportune venue for delving into the am- 
biguous role of students engaged in applied learning. The student in- 
tern assumes a dual role comparable to the graduate teaching assistant. 
Teaching assistants must navigate between several potentially conflict- 
ing roles, such as peer to the students they teach (all are students earn- 
ing grades) grader , mediator between undergraduates and full-fledged 
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faculty. Student interns, whether paid or unpaid, face similar complexi- 
ties in positioning their role within an organization. 

Scholarship that explores applied learning as a critical force could 
investigate the maintenance and disruptions of power relationships 
that infuse experiential learning. Johnston (2007) notes the disconnect 
between the compliant subservience so often rewarded in cooperative 
education ventures, which could include internships, and the encourage- 
ment of questioning and challenging systems of embedded privilege that 
lie at the heart of critical pedagogy. How might scholarship on applied 
learning reconcile the conflicting social pressures to train obedient em- 
ployees while also fostering critical thinkers who question the embedded 
systems of privilege that perpetuate subservience? A study by Carson 
and Fisher (2006) found that 25 percent of students did not demonstrate 
signs of critical thinking in their reflections on the internship experience. 
These students simply described their experiences without questioning 
what they encountered. The researchers note that students might become 
reluctant to engage in critical reflection when simple description poses 
fewer risks. “We recognize that attempting to produce criticality in an 
environment that often asks for the opposite (conformity and adherence 
to the status quo) can be a risky undertaking, one that many students may 
be reluctant to pursue” (Carson & Fisher, 2006, p. 716). To what extent 
can applied learning move toward more critical reflection without jeop- 
ardizing the continuity of campus-community or campus-corporation 
partnerships? 

Another productive path for research would lead researchers toward 
reflecting more deeply on the relationship between applied learning and 
the cultivation of what Benjamin Barber (2004) calls “strong democ- 
racy.” A more substantial body of empirical research could document 
which kinds of applied learning correlate with long-term behavioral 
changes that reflect deep involvement in civic affairs. Theoretical and 
interpretive research could explore how applied learning can build the 
capacity for systemic democratization, perhaps by creating a culture 
of mutual obligation and care as a counterpoint to consumerism and 
commodification (Schwartzman & Phelps, 2002). For example, studies 
could adopt networking theories or diffusion of innovations to determine 
the most effective ways to disseminate the cultural awareness students 
acquire from study abroad. Such investigations might provide insights 
regarding ways to counteract American hegemony and foster more 
civilized intercultural dialogues. 

Future scholarship on applied learning can blaze many promising in- 
tellectual trails. We are pleased to play a role in this exploratory venture 
and invite all scholars and practitioners involved in applied learning to 
join the journey. 
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Applied learning pedagogies—including service-learning, internships/practica, study 
abroad, and undergraduate research—have in common both the potential for significant 
student learning and the challenges of facilitating and assessing that learning, often in 
non-traditional ways that involve experiential strategies outside the classroom as well 
as individualized outcomes. Critical reflection oriented toward well-articulated learning 
outcomes is key to generating, deepening, and documenting student learning in applied 
learning. This article will consider the meaning of critical reflection and principles of good 
practice for designing it effectively and will present a research-grounded, flexible model 
for integrating critical reflection and assessment. 


Applied learning pedagogies share a design fundamental: the nur- 
turing of learning and growth through a reflective, experiential process 
that takes students out of traditional classroom settings. The approach is 
grounded in the conviction that learning is maximized when it is active, 
engaged, and collaborative. Each applied learning pedagogy provides 
students with opportunities to connect theory and practice, to learn in 
unfamiliar contexts, to interact with others unlike themselves, and to 
practice using knowledge and skills. 

Despite the oft-cited maxim that “experience is the best teacher,” 
we know that experience alone can, in fact, be a problematic teacher 
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(Dewey, 1910; Conrad & Hedin, 1990; Hondagneu-Sotelo & Raskoff, 
1994; Stanton, 1990; Strand, 1999). Experiential learning can all too eas- 
ily allow students to reinforce stereotypes about difference, to develop 
simplistic solutions to complex problems, and to generalize inaccurately 
based on limited data. The service-learning student, for example, may 
think that all food assistance programs function exactly like the one at 
which he is working, causing him to make sweeping generalizations 
about the effectiveness of such programs despite widespread variations 
in size, structure, and sources of food and funding. 

In addition, students may not derive the most important or signifi- 
cant learning from their experiences. The undergraduate researcher in 
the physiology lab may be frustrated by the tediousness of the research 
and not appreciate that scientific inquiry is intentionally a slow process 
of trial and error. She may not fully understand why the research ques- 
tions she is investigating are important or how the data she is collecting 
fit into previous findings. 

Students may leave applied learning experiences with little capacity 
to turn learning into improved action. The study abroad student may 
believe he has developed a greater sensitivity to cultures different from 
his own but six months later find himself jumping to conclusions about 
others based on their background or ethnicity. The intern who finds her 
collaborative project frustrating may end up repeating patterns of poor 
teamwork in her next group project. 

Finally, students in applied learning pedagogies may have a vague 
sense of the impact their experiences have had on them but not be fully 
aware of the nature of their own learning, its sources, or its significance. 
They may only be able to describe outcomes vaguely, with phrases such 
as “I learned a lot from working with community members” or “I got 
so much out of living abroad.” The service-learning student may fail to 
understand the different ways in which the classroom and the commu- 
nity present her with learning challenges. The study abroad student may 
be unable to identify specific changes in her attitudes toward others or 
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to articulate what led to the changes. Students may, in other words, miss 
the opportunity to learn about their own learning processes—to develop 
the meta-cognitive skills required for lifelong, self-directed learning that 
applied learning is so well suited to cultivate. 

The students in these examples would all benefit from a process 
of strong reflection, to help them avoid what TS. Eliot (1943) once 
described as having the experience but missing the meaning. Learning— 
and understanding learning processes—does not happen maximally 
through experience alone but rather as a result of thinking about—re- 
flecting on—it. As noted by Stanton (1990), when reflection on experi- 
ence is weak, students’ “learning” may be “haphazard, accidental, and 
superficial” (p. 185). When it is well designed, reflection promotes 
significant learning, including problem-solving skills, higher order rea- 
soning, integrative thinking, goal clarification, openness to new ideas, 
ability to adopt new perspectives, and systemic thinking (Eyler & Giles, 
1999; Conrad & Hedin, 1987). 

However, reflection and its central role in applied learning are often 
misunderstood or seen as unnecessary. The word itself frequently con- 
notes stream-of-consciousness writing, keeping a diary, or producing a 
summary of activities. It can easily be associated with “touchy-feely” 
introspection, too subjective to evaluate in a meaningful way and lack- 
ing in the rigor required for substantive academic work. Dewey (1910), 
one of the early champions of experiential learning, provides a strong 
foundation for re-conceptualizing reflection, defining it as the “active, 
persistent and careful consideration of any belief or supposed form of 
knowledge in the light of the grounds that support it and the further 
conclusions to which it tends” (p. 6). Schén (1983) emphasizes the 
link between reflection and action; he defines reflection as “a continual 
interweaving of thinking and doing” and suggests that what he calls the 
reflective practitioner is one who “reflects on the understandings which 
have been implicit in [one’s] action, which [one] surfaces, criticizes, 
restructures, and embodies in further action” (p. 281). The reflection 
required if applied learning pedagogies are to be maximized as learning 
opportunities is best understood in these terms, as a process of meta- 
cognition that functions to improve the quality of thought and of action 
and the relationship between them. 

When understood in this light and designed accordingly, reflection 
becomes “critical reflection.” It generates learning (articulating ques- 
tions, confronting bias, examining causality, contrasting theory with 
practice, pointing to systemic issues), deepens learning (challenging 
simplistic conclusions, inviting alternative perspectives, asking “why” 
iteratively), and documents learning (producing tangible expressions of 
new understandings for evaluation) (Ash & Clayton, 2009a and 2009b; 
Whitney & Clayton, in press). As we understand it, critical reflection 
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is an evidence-based examination of the sources of and gaps in knowl- 
edge and practice, with the intent to improve both. Designing reflection 
effectively so as to make applied learning educationally meaningful 
first requires that we make clear its meaning as an integrative, analyti- 
cal, capacity-building process rather than as a superficial exercise in 
navel-gazing (Ash & Clayton, 2009b; Whitney & Clayton, in press; 
Zlotkowski & Clayton, 2005). 

A critical reflection process that generates, deepens, and documents 
learning does not occur automatically—trather, it must be carefully and 
intentionally designed. Welch (1999) points out that it is not enough to 
tell students “it is now time to reflect” (p. 1). Eyler, Giles, and Schmiede 
(1996) note that reflection “need not be a difficult process, but it does 
need to be a purposeful and strategic process” (p. 16). Especially given 
how unfamiliar most students are with learning through reflection on 
experience (Clayton & Ash, 2004), they need a structure and guidance 
to help them derive meaningful learning when they are outside the tradi- 
tional classroom setting, otherwise reflection tends to be little more than 
descriptive accounts of experiences or venting of personal feelings. 

This article explores principles of good practice across three steps in 
the design of critical reflection in applied learning: 


1) determining the desired outcomes: learning goals and associated 
objecives, 

2) designing reflection so as to achieve those outcomes, and 

3) integrating formative and summative assessment into the re- 
flection process. 


It then presents a model for critical reflection—the DEAL model—that 
has been explicitly designed to embody these principles and refined 
through several years of research. 

The discussion here is grounded in the conviction that facilitators of 
student learning in applied learning pedagogies are instructional design- 
ers; they make choices throughout the design process that are influenced 
by their goals and constraints and by their students’ abilities as well as 
their own. Designing reflection proceeds best when framed in scholarly 
terms: as a process of experimentation, of continual assessment and 
refinement, of learning with and alongside the students. In other words, 
the designer of applied learning opportunities is best understood as a 
reflective practitioner herself—one who engages in the same critical 
reflection that she expects from her students— thereby improving her 
thinking and action relative to the work of generating, deepening, and 
documenting student learning in applied learning. 
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DETERMINING DESIRED LEARNING OUTCOMES 


Just as with any other intentional design process, designing criti- 
cal reflection requires beginning with the end in mind (Covey, 1989; 
Wiggins & McTighe, 1998). Specifically, it begins with the identifica- 
tion of desired learning outcomes. It then proceeds with the expression 
of learning goals in terms of assessable learning objectives and contin- 
ues to the design and implementation of teaching and learning strategies 
(such as reflection) aligned with those objectives, all the while develop- 
ing assessment strategies that are well-matched to the objectives and to 
the teaching and learning strategies and that can be used to inform future 
revisions of either or both. 

Instructors, as well as the programs that support them, have a range 
of desired learning outcomes that underlie their use of any particular 
applied learning pedagogy (or combination of them). Figure | provides 
a conceptual framework for articulating a categorization of these out- 
comes and the role of critical reflection in advancing them, using service- 
learning as an example. Most instructors use service-learning to help 
their students engage more effectively with the content of the course 
or the perspective of the discipline while also learning about citizen- 
ship and about themselves as individuals. In other words, they use ser- 
vice-learning to help students learn at least in the general categories of 
academic enhancement, civic learning, and personal growth. These cat- 
egories can apply to other applied learning pedagogies as well, along 
with additional ones such as intercultural learning (particularly relevant 


Figure 1: Conceptual Framework for the Role of Reflection in Achieving Categories of Learning 
Goals (service-learning example) 


Components 


academic 
material 


> 


Learning Goal Categories 
critical critical thinking within all categories 


relevant reflection 


service 
personal 
growth 


civic 
learning 
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in study abroad), professional development (especially for internships), 
and research skill development (in undergraduate research). Critical 
thinking might be seen as its own category of outcomes or as a dimen- 
sion of other categories; additional meta-level outcomes related to learn- 
ing processes might include emotional intelligence or the ability to make 
connections between ideas. 

Given the public purposes of higher education (Boyer, 1996; Salt- 
marsh, Hartley, & Clayton, 2009; O’Meara & Rice, 2005; Saltmarsh, 
2005), designers of any applied learning pedagogy might well consider 
civic learning as a relevant category of learning. Battistoni (2002) offers 
thirteen conceptual frameworks for understanding “civic” that are linked 
to various disciplines and thereby suggests a wide variety of ways it can 
be defined, such as in terms of participatory democracy, social justice, 
or an ethic of care. Specific learning goals in this category might relate 
to such issues as change agency, power, privilege, leadership, economic 
and political systems, governmental processes, community organizing, 
and public problem-solving. In light of the multi-faceted nature of this 
category, applied learning opportunities of all types can be designed to 
include it. For example, students involved in undergraduate research can 
consider the social drivers for and implications of both their research 
questions and their process of inquiry; those studying abroad can focus 
attention on the interconnections between local and global issues and on 
the ways culture shapes notions of citizenship; interns can explore the 
roles of corporations as citizens and the range of opportunities to inte- 
grate their professional and civic lives. 

Similarly, applied learning pedagogies often involve interactions 
with others—classmates, mentors, community members, lab partners, 
officemates—and therefore lend themselves readily to learning in the 
general category of collaboration. Associated learning goals might in- 
clude developing students’ abilities to communicate with diverse others, 
make decisions as a group, assess group members’ strengths and weak- 
nesses and allocate responsibility accordingly, handle interpersonal con- 
flict effectively, hold themselves and others accountable to group norms, 
develop shared visions, and monitor progress toward collective objec- 
tives and reach consensus on appropriate changes in their approach. 

As the previous example suggests, learning goals within any one cat- 
egory of learning can often cross into another category—collaboration 
could also be understood as an element of diversity learning, profession- 
al development, personal growth, or civic learning. It is therefore up to 
instructors, program administrators, and/or students to decide how best 
to express the categories of learning and the associated learning goals for 
their particular situation. Because these categories are likely going to be- 
come headings in, for example, assessment reports, particular attention 
should be paid to what best represents the key arenas of learning that are 
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to be cultivated through the pedagogy. Toward that end, it may be help- 
ful to begin by listing more specific goals (such as the ones given above 
for collaboration) and then determining how best to organize them into 
more general categories. Such an activity may be particularly important 
when a group of instructors undertakes instructional design together as 
part of a program or curriculum, so as to make sure that everyone is in 
agreement with and working towards the same desired outcomes. 

Figure 2 provides an example of the use of Venn diagrams to ex- 
press the learning goals associated with various categories of learning 
that might be developed for internships. As the use of the Venn diagrams 
suggests, learning outcomes are often conceptualized as the intersec- 
tion of two or more categories. A Nonprofit Studies curriculum at North 
Carolina State University that is designed with threaded service-learning, 
for example, articulates learning outcomes at the intersection of academ- 
ic enhancement and civic learning in terms of learning goals including: 
aligning mission, methods, and resources; balancing individual interests 
and the common good; moving beyond charity to systemic change; capi- 
talizing on opportunities associated with diversity; and earning the pub- 
lic trust (Jameson, Clayton, & Bringle, 2008). 


Figure 2: Conceptual Framework for the Role of Reflection in Achieving Learning Goals 
(internship example) 
Components of Internships 


academic 
discipline 


Learning Goals of Internships 
(critical thinking within all categories) 


Professional 
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between personal organizations 
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Whether starting with the general categories and working down to 
more specific learning goals within them or starting with learning goals 
and then determining the most useful way to categorize them, developing 
this broad structure to express and organize desired learning outcomes is 
key to undertaking an intentional instructional design process, to com- 
municating the rationales for applied learning to students and colleagues, 
and to structuring assessment strategies and sharing resultant data. This 
structure for thinking about learning outcomes provides an important 
foundation for developing strong approaches to critical reflection. 


FROM LEARNING GOALS TO LEARNING OBJECTIVES 


Once the general categories of learning and their associated learning 
goals have been determined, the instructional designer’s next task is to 
express the learning goals as assessable learning objectives. Goals such 
as “students will learn about project management” (internship), “students 
will understand the challenges facing schools in their attempts to imple- 
ment state and federal education policies” (service-learning), “students 
will appreciate the similarities as well as the differences between their 
home and host cultures” (study abroad), or “students will understand 
the differences between quantitative and qualitative research methods” 
(undergraduate research) are difficult to translate into effective 
pedagogical practice. 

Bloom’s Taxonomy of Educational Objectives (1956) provides a 
foundation for turning learning goals into assessable learning objectives, 
which then drive the rest of the design process. The taxonomy includes 
learning in three domains: cognitive, affective, and psychomotor; this 
discussion refers to the Taxonomy of Educational Objectives in the 
Cognitive Domain. Although modified and re-ordered by some scholars 
in recent years, Bloom et al. originally identified six levels, each with as- 
sociated—and assessable— learning behaviors, as summarized in Table 
1. A central and widely shared, although not universal, tenet of our read- 
ing of the taxonomy is its hierarchical nature—each level builds toward 


Table 1: Bloom’s Taxonomy and Associated Learning Behaviors 


Bloom’s Classification Examples of Learning-Related 
Behaviors 
Knowledge Identify, define, order 
Comprehension Explain describe, restate 
Application Apply, solve, choose 
Analysis Analyze, compare, contrast 
Synthesis Synthesize, develop, propose 
Evaluation Evaluate, assess, judge, critique 
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the next, from simpler to more complex dimensions of reasoning. For 
example, applying an academic concept effectively requires having a 
good understanding of it, which itself involves having basic knowledge 
of the underlying facts or theories. 

Table 2 provides an example, drawn from service-learning, of the use 
of Bloom’s Taxonomy to move from general categories of learning to 
specific learning goals and then to assessable learning objectives. 


Table 2: Using Bloom’s Taxonomy to Move from General Categories of Learning to Specific 
Learning Goals to Assessable Learning Objectives (service-learning example) 


Category: Category: Category: 
Personal Growth Civic Learning Academic Enhancement 
Learning Learning Goal: Learning Goal: Learning Goal: 
Objective | Students will consider Students will become Students will understand 
Level ways to refine their more effective change the Stages of Change 
skills agents model 
LO1: Identify a particular skill of Identify the collective Identify the Stages of Change 
Identify yours that you need to develop | objectives at stake and the model. 
further. approach you or others took 
toward meeting them. 
Explain the skill so that Explain the objectives and the Explain the Stages of Change 
Lo2: someone who does not know approach you and / or others model so that someone not in the 
Expl ‘ you can understand it. took toward meeting them so course can understand it. 
xpiain that someone not involved can 


understand. 
Apply your understanding of | Apply your understanding of Apply your understanding of the 
Lo3: this skill in the context of your | the approach in the context of Stages of Change model in the 
Appl service-learning experience the objectives at stake. context of the experience. 

Pply and (as applicable) in other 
areas of your life. 


Analyze the sources of this Analyze the approach in light of | Analyze the similarities and 
Lo4: skill in your life. alternatives. differences between the Stages of 
Anal . Change model as presented in the 
nalyze text and as it emerged in the 
community. 
Develop the steps necessary to | Develop the steps necessary to | Develop an enhanced 
improve upon this skill inthe | make any needed improvements | understanding of the Stages of 
LO 5: short term, in your service- in your / their approaches Change model in light of the 
Synthesize | learning activities and (as (and/or in the objectives) in the | experience. 
applicable) in other areas of short term. 
your life. 
Evaluate your strategies for Evaluate your / their Evaluate the completeness of 
Los: refining your skills over the approaches in terms of the your understanding of the Stages 
Eval 3 long term. prospects for long-term, of Change model and of its use in 
value sustainable, and/or systemic the community. 
change. 


Using Bloom’s Taxonomy in this way, to achieve a high level of 
clarity regarding desired learning outcomes and to express them in as- 
sessable language, enables instructors to design reflection that targets 
learning objectives in developmentally-appropriate ways, building 
toward the highest level of learning deemed appropriate in any given 
instance. The learning objectives thus become both the road map that 
guides the design of reflection activities and the basis for determining 
whether the intended destination has been reached and adequately ex- 
pressed in the products of reflection. 
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DESIGNING REFLECTION TO 
ACHIEVE DESIRED LEARNING 


Effectively designing critical reflection involves making a series of 
choices that are informed by the desired learning outcomes as well as 
by the opportunities and constraints that come with the specific context 
in which applied learning is being implemented and by the abilities of 
the participants. These choices produce an overall reflection strategy 
or over-arching structure that may combine various reflection activities 
or mechanisms—such as journal entries, online chat sessions, poster 
presentations, worksheets, or discussion sessions. Questions such 
as those in Table 3 can help guide the design of reflection strategies 
and mechanisms. 

The result of such intentional design work is a customized plan that 
integrates critical reflection into the core of applied learning experiences. 
This plan may be maximized by designing the reflection strategy such 
that individual reflection mechanisms build on one another cumulatively, 
so that students learn how to learn through reflection as well as improve 
the quality of their learning and their practice over time. Table 4 sum- 
marizes a body of principles of good practice that has emerged to support 
the instructional designer in making the choices that produce high qual- 
ity reflection strategies and mechanisms. 


Table 3: Questions to Guide the Design of Reflection Strategies and Mechanisms 


Reflection Strategies 


When and how often will reflection occur? 
Before, during, and after the experience? 
Will students reflect iteratively such that reflection builds on itself over time? 


Where will reflection occur? 
In or outside the classroom? 


Who will facilitate and/or particpate in reflection? 
Instructors, members of the community or workplace, peers? 
How will feedback be provided and/or reflection products graded? 
What is the relationship between amount of feedback and level of expected outcomes? 
What is the relationship between the reflection products and the overall grade? 
Reflection Mechanisms 


Toward what specific learning goals and objectives will the particular activity be guided? 
medium will be used for the activity: written assignments, worksheets, spectrum activities, 
photographs, videos, games, drawings, online forums, in-class discussion, out-of-class reflection 
sessions, concept maps, etc.? 

What prompts will be used to guide the activity? 

What products will demonstrate the learning the activity generates: essays, PowerPoint or poster 
presentations, oral exams, etc.? 

Note that in a critical reflection process, the products used to demonstrate learning are in 
many cases the same as the medium used to generate it 

What criteria will be used to assess the learning so demonstrated ? 


S 
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Table 4: Characteristics of High Quality Reflection 


High Quality Reflection ... 
Eyler et al. (1996) | is continuous (ongoing) 


is connected (with assignments and activities related to and building on one 
another and including explicit integration with learning goals and academic 
material) 

is challenging (including in terms of the expectation that students take 
responsibility for their own learning) 


is contextualized (to the community setting and broader public issues and 
to the students’ own particular roles) 


Bringle & Hatcher | links experience to learning 
(1999) 


is guided 


occurs regularly 
involves feedback to the learner to enhance the learning 


helps clarify values 


Zlotkowski & is oriented toward specific learning objectives 
Clayton (2005) is integrative 


is assessed in terms of critical thinking 


includes goal setting 


generates change in the learner’s life 


Each of these sets of characteristics of high quality critical reflection 
includes explicit linkage to desired learning outcomes, and Bloom’s Tax- 
onomy provides a structure to facilitate the design of reflection accord- 
ingly. The example reflection activity provided in Figure 3 demonstrates 
the design of reflection prompts—for the learning goal of understand- 
ing strengths and weaknesses, in the category of personal growth—that 
guide students step-by-step to ever-higher levels of reasoning through 
prompts that are explicitly structured in accordance with the levels of 
the taxonomy. 

A focus on critical thinking is a key characteristic of critical reflec- 
tion. The reflection guided by the prompts in Figure 3 can progress to 
ever-higher levels of reasoning but do so poorly, in an illogical, unclear 
way that is uninformed by consideration of multiple perspectives and 
that fails to engage with the true complexity of the issues. Critical think- 
ing, as outlined by Paul and Elder (2002), is based on universal intellec- 
tual standards that include accuracy, clarity, relevance, depth, breadth, 
logic, significance, and fairness. Many of the potential shortcomings 
of reflection described in the introduction—treinforcing stereotypes, 
generalizing inappropriately on the basis of limited data, missing the 
most significant learning in an experience—are indicative of and result 
from poorly developed critical thinking abilities. Providing guidance 
in this area is, therefore, a necessary corollary to the use of hierarchi- 
cal learning objectives in the design of critical reflection (Ash, Clayton, 
& Atkinson, 2005). Table 5 provides an overview of the standards of 
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Figure 3: Sample Bloom-based Reflection Mechanism (undergraduate research example) 


According to Parker Palmer (2000), “limitations and liabilities are the flip side of our 
gifts ... a particular weakness is the inevitable trade-off for a particular strength.” 
There is nothing “wrong” with us that we need to “fix,” he suggests. Rather, we are 
who we are; sometimes our personal characteristics serve us well (and we think of 
them as strengths), and sometimes they serve us ill (and we think of them as 
weaknesses) The attempt to “fix” our liabilities will inevitably alter their “flip side” 
gifts as well. (pp. 52-53). 


Individually and in writing ... 

Identify and explain a personal characteristic that you tend to think of as a weakness 
in your role as a researcher 

Apply Palmer’s distinction to this characteristic: What gift or strength could be the 
“flip side” of this “weakness”? 


Discuss with a partner ... 

Compare and contrast a research-related situation in which the “weakness” emerged 
and one in which its “flip side” strength emerged. Why do you think each 
emerged as it did and what were the consequences? 

If Palmer is correct regarding the relationship between our gifts or strengths and our 
limitations or liabilities, what do you think are the implications for your 
approach to personal and professional development as a researcher? 


Individually and in writing ... 
Critique Palmer’s distinction: Do you agree with him? Why or why not? What, if 
anything, would you change in his thinking? 


critical thinking (with the addition of integration and writing quality), 
along with prompting questions that can be used by students themselves 
to improve the quality of their reasoning and by peers and/or instructors 
as feedback on reflection products. 

Using these tools together—designing reflection mechanisms 
through the use of hierarchical learning objectives and improving the 
quality of thinking at each of the levels of reasoning through the use of 
critical thinking standards—will help to generate and deepen learning in 
an applied learning environment. The products of such intentionally de- 
signed reflection, in turn, document learning for purposes of grading or 
research as well as for student use in guiding future thinking and action. 


INTEGRATING FORMATIVE AND SUMMATIVE 
ASSESSMENT INTO THE REFLECTION PROCESS 


Designing an intentional approach to critical reflection in applied 
learning also involves the development of an assessment strategy. Just 
as reflection is much more effectively implemented not only at the end 
of an applied learning course or project but throughout, so too is assess- 
ment more valuable when it is designed from the beginning and is itself 
evaluated and modified as needed throughout. 
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Assessment can be designed for summative purposes and used at 
the end of a process to measure and document outcomes, and it can be 
designed for formative purposes and used during a process as a way to 
continuously improve both the process and the outcomes. A summative 
assessment process that is grounded in well-articulated learning objec- 
tives can be used both to grade student products and to report outcomes 
at program or curriculum levels. Summative assessment in the form of 
grading generally involves judging the degree to which students have 
met the learning objectives. Such assessment can be standards based 
and therefore measure the ultimate attainment of an objective at the 
end of the experience, or it can be based on improvement and therefore 
measure change over time. A related design choice that often emerges 
at the program or curriculum level is whether the ultimate attainment 
or the change over time is to be assessed within a single course or ap- 
plied learning project, across a sequence of courses or projects, or both. 
Instructors and/or administrators need to decide on the form summative 
assessment reports should take, in light of the uses to which they will 
be put and the audiences for whom they are intended. For example, will 
the assessment be expressed quantitatively, such as the percentage of 
students whose reflection on experience demonstrates fulfillment of the 
desired outcomes, or will the report provide qualitative information with 
examples of student learning outcomes, or both? 

Faculty and students using applied learning pedagogies will find val- 
ue in formatively assessing both learning and the teaching and learning 
process and programs that generate it. Formative assessment is increas- 
ingly recognized as key to effectively designing teaching and learning. 
As noted by the National Research Council (2001), “Students will learn 
more if instruction [in this case, reflection in applied learning] and as- 
sessment are integrally related. [P]roviding students with information 
about particular qualities of their work and what they can do to improve 
it is crucial for maximizing learning” (p. 258). Feedback combined 
with opportunities to apply it (e.g. through revision of their work) is 
an approach to formative assessment that helps students learn not only 
content, but meta-cognitive skills as well—in this case, learning how to 
learn through the often unfamiliar process of critical reflection. 

Formative assessment can also be used to check the reflection process 
against the learning outcomes it generates so as to refine both the learn- 
ing goals and objectives and the reflection strategies and mechanisms 
designed to meet them. Instructors might review student products criti- 
cally not only in order to provide helpful feedback to improve students’ 
thinking but also to gauge the effectiveness of their own design (e.g., the 
clarity of the reflection prompts) and to provide themselves with feed- 
back to improve it. Such formative assessment also provides valuable 
feedback to instructors regarding, for example, concepts or skills that 
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prove difficult for students to grasp; such information can inform discus- 
sion of how these concepts or skills are taught in the courses or programs 
associated with or prerequisite to the applied learning activity. 

Having sorted through the various purposes of assessment, the de- 
signer of applied learning pedagogies faces additional choice points 
related to how assessment is implemented, including the nature of the 
products or evidence that will be examined. Will assessment involve ex- 
tra activities that are not related to the learning process (for example, pre 
and post questionnaires) or assignments and products that are already 
part of the course or project (for example, reflection products or essays)? 
In a questionnaire, students might be asked to what degree they think 
they have met the learning objectives of their applied learning experi- 
ence; in a course-embedded assignment, students would be asked to re- 
spond to a prompt or prompts, and the resultant product would be evalu- 
ated against the objectives. Practitioner-scholars such as Eyler (2000) 
suggest that the former often confuses student satisfaction with student 
learning and therefore call for the development of approaches that sup- 
port students in doing the latter. In addition, a course-embedded process 
is generally less time-consuming, for both students and instructors, than 
the interview, focus group, or portfolio methods often used (Eyler & 
Giles, 1999; Gelmon, Holland, Driscoll, Spring, & Kerrigan, 2001) and 
requires more intentional integration of assessment with the teaching and 
learning process. 

Another issue is the determination of criteria that will be used to eval- 
uate the products that demonstrate learning. In other words, what will 
be the indicators of the degree to which the student has met the learning 
objectives or of the quality of learning outcomes? The creation of a ru- 
bric that expresses varying levels of quality or mastery, from novice to 
expert or from under-developed to excellent, can be extremely helpful in 
guiding this process. For example, if the objective is for interns to be able 
to determine the appropriate approach to a particular workplace situation 
(e.g., a team member not pulling her weight), a rubric in which responses 
are categorized by degree of sophistication and/or efficacy could be used 
for assessment. If an objective is meta-cognitive and/or is unique to the 
students as individuals and their particular experiences (for example, 
that students are able to evaluate a personal strength or weakness in light 
of their professional goals), then a rubric based on Bloom’s Taxonomy 
that expresses levels of reasoning may be in order. The Standards of 
Critical Thinking described earlier can also be turned into a rubric (see 
Table 6, for example) that can be used to assess quality of reasoning. 

An integrated approach to assessment and reflection includes using 
the same set of objectives and standards and tools to generate learning 
(through reflection prompts), to deepen learning (through formative as- 
sessment or feedback), and to document learning (through summative 
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Table 6: Critical Thinking Rubric [excerpts] 
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completely lacking (1) | under-developed (2) good (3) excellent (4) 
Consistently makes Makes several Usually but not always Consistently makes 
inaccurate statements inaccurate statements makes statements that statements that are 
and/or fails to provide and/or supports few are accurate and well- accurate and well- 
supporting evidence for statements with evidence | supported with evidence | supported with evidence 
Accuracy _| claims 
Consistently fails to Only occasionally Usually but not always Consistently provides 
provide examples, to provides examples, provides examples, examples, illustrates 
Clarity illustrate points, to define | illustrates points, defines | illustrates points, defines | points, defines terms, 
terms, and/or to express terms, and/or expresses | terms, and/or expresses and/or expresses ideas in 
ideas in other ways ideas in other ways ideas in other ways other ways 
Fails to address salient Addresses few of the Addresses some but not Thoroughly addresses 
questions that arise from | salient questions that all of the salient questions | salient questions that arise 
statements being made; arise from statements that arise from statements | from statements being 
Depth consistently over- being made; often over- | being made; rarely over- | made; avoids over- 
simplifies when making simplifies when making | simplifies when making simplifying when making 
connections; fails to connections; considers connections; considers connections; considers the 
consider any of the little of the complexity some but not all of the full complexity of the 
complexities of the issue | of the issue full complexity of the issue 
issue 
Ignores or superficially Gives minimal Gives some consideration | Gives meaningful 
considers alternative consideration to to alternative points of consideration to 
points of view and/or alternative points of view and/or alternative points of view 
Breadth interpretations view and/or interpretations and makes | and/or interpretations and 
interpretations and some use of them in makes very good use of 
makes very limited use __| shaping the learning them in shaping the 
of them in shaping the being articulated learning being articulated 
learning being 
articulated 
Consistently represents Occasionally represents | Often but not always Consistently represents 
Fairness others’ perspectives ina | others’ perspectives ina | represents others’ others’ perspectives with 
biased or distorted way biased or distorted way | perspectives with integrity (without bias or 
integrity distortion) 
Modified source: Paul, R.P. & Elder, L. 2001. The Miniature Guide to Critical Thinking. The Foundation for Criti- 


cal Thinking. Santa Rosa, CA. www.criticalthinking.org] 


assessment or grading and reporting outcomes). Reflection prompts 
based on Bloom’s Taxonomy can both guide students to desired levels of 
reasoning and determine the level of reasoning they have attained. Criti- 
cal thinking standards can be used as both a formative guide to improve 
student reasoning and a summative tool to evaluate its quality in the end. 
Making visible such integration of reflection and assessment is key in 
helping students become increasingly aware of and responsible for their 
own learning processes. 

The creation of an assessment strategy is as important as the articula- 
tion of the learning goals and associated objectives, and all should be 
developed in parallel during the design of the reflection activities. Trying 
to assess a learning goal that has not been articulated as an assessable 
objective (e.g., “students will understand ...,” “students will appreciate 
....” “students will learn about ...”) is usually an exercise in frustration. 
A reflection mechanism that is not mapped to learning objectives is often 
a missed opportunity for maximized learning as well as a hindrance to 
using reflection products to assess learning. And an objective that ex- 
presses desired learning that cannot be achieved through the pedagogy 
in question, much less assessed, should, like all of the above, send the 
designer back to the drawing board. 
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THE DEAL MODEL FOR CRITICAL REFLECTION 


An example of an approach to critical reflection explicitly designed 
in accordance with the principles of good practice discussed above is 
the DEAL Model for Critical Reflection (Ash & Clayton, 2004; Ash & 
Clayton, 2009a, 2009b)—the product of a multi-year scholarship of 
teaching and learning project involving students and faculty from a va- 
riety of disciplines. Originally developed in the context of service-learn- 
ing, DEAL has been used across a range of traditional and experiential 
pedagogies; in K-12, undergraduate, and graduate courses and curricula; 
and in co-curricular as well as professional training settings. 


The DEAL model consists of three sequential steps (see Figure 4): 


1. Description of experiences in an objective and detailed manner; 

2. Examination of those experiences in light of specific learning 
goals or objectives; and 

3. Articulation of Learning, including goals for future action that 
can then be taken forward into the next experience for im- 
proved practice and further refinement of learning. 


Figure 4: Schematic Overview of the DEAL Model for Critical Reflection 


Describe Engage in 
Engage in experience expenence and 
experience objectively test learning 
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Each step of this model requires specific prompts, which provide the 
guidance necessary for students to engage in the oftentimes counter-nor- 
mative activity of developing their own learning rather than reproducing 
what their instructors have taught them (Clayton & Ash, 2004; Howard, 
1998). The discussion that follows summarizes each step in the DEAL 
model and provides sample prompts. 


DESCRIBE 


Objective, detailed description of an experience provides a strong 
foundation for meaning-making in the critical reflection process; it is a 
way to make the experience present and to ensure that students have ac- 
cess to all relevant aspects of it as they engage in reflection. This step is 
not as simple as it might appear, as students often prefer to jump straight 
into interpretation. It is also easy to overlook or under-value the details 
that are often most significant, so enhanced skills of mindfulness and at- 
tentiveness are often required for—and developed by—this step. Reflec- 
tion prompts associated with the Describe step ask students to address 
such issues as when and where the experience in question took place, 
who was and was not present, what they and others did and did not do, 
what they saw and heard, and so on. 


EXAMINE 


The DEAL model is explicitly designed to move students beyond 
summarizing their experiences, which all too often results when a re- 
flection activity is assigned, into meaning-making. In the second step of 
DEAL, prompts that help students Examine their experiences are linked 
to the desired learning outcomes—whether expressed as learning goals 
or, in a more assessable fashion, as learning objectives—within each 
category of learning. Table 7 provides examples of prompts drawn from 
learning goals in the general category of civic learning; some instruc- 
tors may prefer to develop Examine prompts from learning goals such 
as these rather than from assessable objectives when, for example, the 
intent is to stimulate questions or surface issues for further discussion 
rather than to evaluate students’ reasoning. 


ARTICULATE LEARNING 


The third step of the DEAL model supports students in 
Articulating the Learning that the two previous steps have begun to 
generate, while providing further guidance in continuing to expand and 
deepen that learning. It helps them capture their learning in such a way 
as to be able to act on it and thereby improve the quality of their learn- 
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Table 7: DEAL Model Sample “Examine” Prompts Based on Learning Goals in the General 
Category of Civic Learning 


Learning Goals 


Sample Examine Prompts 


Students will explore 
the dynamics of change 
agency 


What was I / someone else trying to accomplish? 


In taking the actions I / they did, was the focus on symptoms of problems 
or causes of problems? 


Was the focus (symptom or cause) appropriate to the situation? 


How might I / they focus more on underlying causes in the future? 


Students will learn 
about power and 
privilege 


In what ways did differentials in power and privilege emerge in this 
experience? 


What are the sources of power and privilege in this situation? 


Who benefits and who is harmed? 


Students will appreciate 
the tension between 


What is in the interest of the common good in this situation? What is in 
the interest of (whose) individual interests or rights? 


individual interests and 


In what ways is the individual good (mine / others) linked to and/or 
the common good 


contrary to the common good? 


What trade-offs between them are involved? Who made the trade-offs? 
Were the trade-offs made appropriate or inappropriate and why? 


ing and their future actions. It consists of four prompts: (a) What did I 
learn? ; (b) How did I learn it?; (c) Why does it matter?; and (d) What 
will I do in light of it? The DEAL model thus does not begin but rather 
ends with the question “What did you learn,” in accordance with the 
understanding of reflection as the component of applied learning that 
generates learning. 

The general structure provided by the DEAL model can be used to 
guide critical reflection online, in an oral discussion, in a written journal 
entry or essay, or in any combination of mechanisms. For example, De- 
scription might be done online by each student individually, Examina- 
tion orally by a group of students, and Articulation of Learning as a writ- 
ten essay. The DEAL model can be used to structure “light” reflection, 
as in a 30 minute in-class activity or an online chat that produces simple 
(e.g., four sentence) Articulated Learnings. 


A BLOOM-BASED USE OF DEAL 


DEAL can also guide more in-depth critical reflection that targets 
higher order reasoning and critical thinking through prompts that are 
tied directly to hierarchical learning objectives. Such an approach might 
be used not merely to stimulate questions and surface issues for further 
discussion, as in the goal-based example in Table 7 above, but also to 
support students explicitly in developing reasoning abilities and to assess 
the quality of their reasoning. 

In a particularly comprehensive version of the DEAL model (Ash & 
Clayton, 2009a, 2009b), designed to facilitate student reasoning all the 
way up to the level of evaluation in Bloom’s Taxonomy, the Examine 
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and the Articulate Learning steps each have two parts. After students 
Describe an experience, they surface one or more key ideas for further 
thought in Examine Part I and then take one of those ideas from identi- 
fication and explanation through application and to analysis in Examine 
Part II. In Part I of the Articulate Learning step they synthesize a new 
understanding of the key idea and evaluate changes in their thinking, and 
in Part I they evaluate the written expression of that thinking and revise 
it as needed. 

For example, in the category of personal growth, Examine Part I 
might include some or all of the prompts in Table 8, which are oriented 
toward the learning goals that comprise this category and which encour- 
age students to focus on their own particular personal characteristics. 

Then Part II of the Examine step might use prompts such as those in 
Table 9—specifically mapped to Bloom-based learning objectives up to 
the level of analysis—to support students in developing their thinking 
about that characteristic further. 

The Articulate Learning step then supports students in re-thinking 
and extending the thinking from the Examine step, to create a more 
meaningful and fully thought out reflective essay, moving them through 
Synthesis and Evaluation with additional sub-prompts and supporting 
them in documenting all six levels of reasoning in Bloom’s Taxonomy. 
Continuing with the example in the category of personal growth, Part I 
of this step includes the expanded prompts represented in Table 10. 


Table 8: Bloom-based Version of DEAL: Sample “Examine” Part I Prompts (Personal Growth 
Category) 


Examine Part I (Personal Growth): Sample Prompts to Surface a Personal 
Characteristic 

What assumptions or expectations did I bring to the situation? How did they affect what I 
did or didn’t think, feel, decide, or do? To what extent did they prove true? If they did not 
prove true, why was there a discrepancy? 
How did this experience make me feel (positively and/or negatively)? How did I handle 
my emotional reactions? Should I have felt differently? Why or why not? 
How did I interpret the thoughts, feelings, decisions, and/or behaviors of others What 
evidence do I have that my interpretations were or were not accurate? 
In what ways did I succeed or do well in this situation (e.g., interacting with others, 
accomplishing tasks, handling difficulties) and what personal characteristics helped me to 
be successful (e.g., skills, abilities, perspectives, attitudes, tendencies, knowledge)? In 
what ways did I experience difficulties (e.g., interacting with others, accomplishing tasks) 
and what personal characteristics contributed to the difficulties (e.g., skills, abilities, 
perspectives, attitudes, tendencies, knowledge)? 
How did this situation challenge or reinforce my values, beliefs, convictions (e.g., my 
sense of right and wrong, my priorities, my judgments)? My sense of personal identity 
(e.g., how I think of myself in terms of gender, sexual orientation, socioeconomic status, 
age, education level, ethnicity, nationality, mental/physical health)? 
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In Part II of the Articulate Learning step, students are asked to evalu- 
ate their written products using a checklist, which includes the standards 
of critical thinking, and to rewrite their “I learned that” statement as 
needed to ensure that it expresses the highest level of learning they have 
achieved. 


Table 9: Bloom-based Version of DEAL: Sample “Examine” Part II Prompts (Personal Growth 
Category) 


Examine Part II (Personal Growth): Prompts to Develop Understanding of a Personal 
Characteristic Using Bloom’s Taxonomy 


Identify What personal characteristic are you coming to understand better as a result of 
reflection on your applied learning experiences? 

Explain Explain the characteristic so that someone who does not know you would 
understand it. 

Apply How does / might this characteristic positively and/or negatively affect your 


interactions with others, your decisions, and/or your actions in your applied 
activities and (as applicable) in other areas of your life? 

Analyze What are the possible sources of / reasons for this characteristic? How does 
your understanding of these sources / reasons help you to better understand 
what will be involved in using, improving, or changing this characteristic in the 
future? 


Regardless of how it is implemented—written or oral, individual or 
collaborative, lightly or in-depth—the DEAL model offers students the 
opportunity to use writing or speaking as vehicles for learning rather 
than as expressions of learning after it has already occurred (Clayton 
& Ash, 2004). Generating their own learning in this way is yet another 
counter-normative aspect of critical reflection on experience and, as sug- 
gested in the set of characteristics of high quality reflection in Table 4, 
students will benefit from feedback on their thinking, with associated 
opportunities to revisit and revise (e.g., through application of the Stan- 
dards of Critical Thinking presented in Tables 5 and 6) to maximize the 
quality of their learning. 

In addition, the development of a critical reflection model such as 
DEAL facilitates scholarly work relative to teaching and learning in an 
applied learning pedagogy, helping instructors improve the former to en- 
hance the latter. For example, DEAL and its associated rubrics (includ- 
ing the critical thinking rubric in Table 6) were used to examine changes 
in students’ critical thinking and higher order reasoning abilities across 
drafts of a single reflection product and over the course of a semester, as 
well as across the categories of academic enhancement, civic learning, 
and personal growth in several service-learning enhanced classes (Ash 
et al., 2005). Building on this work, Jameson et al. (2008) modified the 
DEAL reflection prompts and rubrics for application across the course 
sequence of a Nonprofit Studies minor, investigating changes in stu- 
dents’ critical thinking and reasoning abilities across the learning goals 
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Table 10: Bloom-based Version of DEAL: “Articulate Learning” Part I Prompts (Personal 
Growth Category) 


1. What did I learn? 


e Identify and explain (so that someone who doesn’t know you can understand it) a 
personal characteristic that you are beginning to understand better 

e Express the learning in general terms, not just in the context of the experience, so 
that it can be applied more broadly to other areas of your life (personally or 
professionally) and help you in your ongoing personal growth process 

e Introduce a judgment regarding whether the characteristic serves you well (and 
thus needs to be capitalized on) or poorly (and thus needs to be changed) — or both 


2. How did I learn it? 


e Clearly connect the learning to your specific applied learning activities so that 
someone who was not involved would understand, including discussion of the 
positive and negative impacts of the personal characteristic 


3. Why does it matter? 


¢ Consider how the learning has value over the short and long term, both in terms of 
your applied learning activities and in terms of your life more generally 


4. What will I do in light of it? 


e Set specific goals and assessable goals (that you could come back to and check on 
to see if they are being met) relative to this learning over the short and long term 

¢ Consider the benefits and challenges associated with fulfilling these goals, 
especially in light of the sources of or reasons for the characteristic 


of five leadership challenges facing the nonprofit sector. McGuire et al. 
(2009) examined critical thinking demonstrated in Articulated Learnings 
produced by students in multiple disciplines using a variety of assign- 
ment and feedback-revision formats. 

The DEAL model and its associated rubrics therefore demonstrate 
the intentional design of critical reflection: identifying desired student 
learning outcomes, articulating them as specific goals and as assessable 
learning objectives, and then crafting an integrated reflection and assess- 
ment approach around them. In addition to providing tools needed to 
generate, deepen, and document student learning, DEAL facilitates in- 
vestigation of the learning processes (Clayton, Ash, & Jameson, 2009). 


CONCLUSION 


It is our hope that our work can serve as a model for faculty, staff, and 
students as they seek to design reflection associated with applied learn- 
ing opportunities, courses, and programs. Our individual and collective 
learning as practitioner-scholars across the field of applied learning can 
be enhanced through a scholarly approach to the instructional design pro- 
cess. In turn, it can contribute to advancing the academy’s understanding 
of both how our students think and how we can support them in learning 
to think more deeply and with greater capacity for self-directed learning. 
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The Victorian Certificate of Applied Learning (VCAL) is a very successful senior second- 
ary school qualification introduced in the Australian state of Victoria in 2002. Applied 
learning in the VCAL engages senior students in a combination of work-based learning, 
service-learning, and project-based learning and aims to provide them with the skills, 
knowledge, and attitudes to make informed choices regarding pathways to work and 
further education. The program has enjoyed rapid growth and its system-wide adoption 
by Victorian secondary schools, Technical and Further Education (TAFE) institutions, 
Registered Training Organizations (RTOs), and Adult and Community Education (ACE) 
providers has broadened significantly the range of senior schooling pathway options for 
young people. This paper will examine reasons for developing an applied learning senior 
secondary certificate and its rapid growth in Victoria since 2002. The authors draw on a 
number of case studies to profile the unique nature of applied learning in the VCAL, includ- 
ing its dimensions of service learning, work-based learning, and project-based learning. 
These case studies are also used to discuss a number of implications that have emerged 
from the use of applied learning in the VCAL, including approaches to teaching and as- 
sessment that will support applied learning and the development of new partnerships be- 
tween VCAL providers and community partners. Finally, the paper considers significant 
implications the VCAL has created for teacher education in Victoria by discussing the 
new Graduate Diploma of Education (Applied Learning) developed by Deakin University. 
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Improving the education participation rates of young people has be- 
come a significant social and economic goal for successive Australian 
and international governments (OECD, 2001). Such a strategy is now 
widely understood to improve a nation’s economic competitiveness 
as well as address issues of social inclusion and reduction of poverty 
(Access Economics, 2005; Billett & Seddon, 2004; Considine, 2006; 
Lamb & Rice, 2008). To achieve these increased participation rates there 
has emerged demand for new policies and practices in education and 
training emphasizing the need to increase the quality and relevance of 
learning to the lives of young people, particularly in their middle and 
post-compulsory years of schooling (UNESCO, 2008). Education insti- 
tutions in many countries now place much more significance on the need 
for high quality alternative education opportunities (Aron, 2006), such as 
the “hands-on” or “applied” approaches to students’ learning. Alongside 
these new directions in education there also emerge new questions for 
educators in secondary schools and higher education: (1) What do these 
high quality programs look like and what is the nature of their applied 
learning? and (2) What are the implications for teachers’ professional 
practice and how can applied learning be used in higher education to pre- 
pare the new “applied learning teachers” required in secondary schools? 

This paper explores these questions in two stages. In the first in- 
stance, the authors draw on current education participation trends in 
Australia to examine the development of the Victorian Certificate of 
Applied Learning (VCAL) currently being offered to senior high school 
students in Victoria, Australia. They then provide an outline of the 
VCAL in its current form and draw on two case studies to discuss the 
nature of the applied learning in the program. The second part of the 
paper discusses implications for higher education programs targeting 
the preparation of pre-service teachers in Victoria. The authors draw on 
the recent development of a Graduate Diploma of Education (Applied 
Learning) which seeks to utilize applied learning in higher education to 
prepare pre-service teachers for teaching in the VCAL. 


AUTHOR NOTE: Damian Blake, Ph.D., senior lecturer in applied learning in 
the School of Education, Deakin University Waurn Ponds Campus, Geelong 
Victoria Australia; David Gallagher, Victorian Curriculum and Assessment Au- 
thority (VCAA), East Melbourne Victoria Australia. Correspondence concern- 
ing this article should be addressed to: Damian Blake, School of Education, 
Deakin University Waurn Ponds Campus, Geelong Victoria Australia 3217 (email: 
damian.blake @deakin.edu.au) or David Gallagher, Senior Manager for the Vic- 
torian Curriculum and Assessment Authority (VCAA), 41 St Andrews Place, East 
Melbourne Victoria Australia 3002 (email: gallagher.david j@ edumail.vic.gov.au) 
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LINKING EDUCATION TO THE ECONOMY 


Since the early 1990s many governments around the world have 
linked education policy to economic reform. In Australia, for example, 
some have argued: “Education is increasingly becoming the ‘engine 
room’ of modern economies...A more educated worker is a more pro- 
ductive worker, and investment in education provides a pool of more 
skilled labor” (Access Economics, 2005, p. 8). Like other countries, 
much of the Australian education policy reform agenda has aimed to 
increase the proportion of young people who complete at least 12 years 
of schooling. 

Each year approximately 80% of young Australians complete a level 
of education and training equivalent to 12 years of schooling (Australian 
Bureau of Statistics, 2007). Research conducted by the Organization 
for Economic Co-operation and Development (OECD) estimates this to 
be lower than other OECD countries, such as 88% in the United States 
and Canada, 91% in Germany, and 94% in Korea (OECD, 2008). Mod- 
eling based on Australian rates of participation in education suggests 
that boosting the proportion of young people completing school or an 
apprenticeship to 90% by 2010 would increase workforce numbers by 
65,000, boost economic productivity, and expand the economy by more 
than nine billion Australian dollars (Access Economics, 2005; Applied 
Economics, 2002). It is therefore not surprising that like other states in 
Australia, the Victorian Government has set the goal that by 2010 it ex- 
pects that 90% of all young people will have achieved Year 12 or its 
equivalent (Kirby, 2000; Kosky, 2002; Lamb & Rice, 2008). 


A DISCOURSE OF RISK AND “EARLY SCHOOL LEAVERS” 


A discourse of “risk” has now emerged alongside the remaining 20% 
of “early school leavers” and students who remain in school considered 
to be “at risk” of leaving. Contributing to this discourse of risk is the 
broader realization that completing 12 years of schooling significantly 
improves life transitions beyond school and reduces a young person’s 
chances of falling into long-term unemployment (DSF, 2006; Lamb & 
Rice, 2008; Woods, 2007). 


Students who do not complete Year 12 or its equivalent are more 
likely to become unemployed, stay unemployed for longer, have 
lower earnings, and over the course of their lives, accumulate less 
wealth, a problem that will only increase with time as employers 
seek a more highly skilled workforce. The connection is simple — 
retention, engagement and higher levels of education open up broad- 
er opportunities that lead to better personal, social and economic 

outcomes. 
(Lamb & Rice, 2008, p. 2) 
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However, this discourse of risk is not limited to the personal risks 
faced by young people who are “early school leavers” and is also re- 
flected in perceived risks to wider society. In the Australian state of New 
South Wales (NSW), for example, the Bureau of Crime Statistics and 
Research argues: “If long term unemployed males aged between 15-24, 
continued to the end of senior high school the reduction in break, enter 
and stealing over the course of a year would amount to almost 15 per 
cent” (Chapman, Weatherburn, Kapuscinski, Chilvers, & Roussel, 2002, 
p. 10). 

Policy aimed at improving participation in the senior years of school- 
ing therefore addresses the social risks associated with potential exclu- 
sion of young people from the important institutions of wider society, 
and the potential political risks created by perceptions of disenfranchised 
youth. The stakes are considered to be high for both “early school leav- 
ers” and their communities, creating a new moral imperative for schools 
to improve further young people’s participation in the senior years 
of schooling. 


A DEMAND FOR NEW “ALTERNATIVES” AND THE VCAL 


Increasing retention rates in Australian schools since the 1990s (Aus- 
tralian Bureau of Statistics, 2007) have already created new challenges 
for schools and teachers as they attempt to accommodate the greater 
diversity of senior students’ pathway interests and adjust to changing 
“learning temperaments” in their classrooms (Henry & Grundy, 2003). 
Schools and teachers are now being challenged to confront the many 
reasons young people give for leaving school early, including: a lack of 
perceived relevance of school; a desire to work and pursue vocational 
pathways; limited experiences of success in academic programs; a re- 
duced tolerance for typical school rules; and problematic relationships 
with teachers (Blake, 2007). 

Despite the rapid expansion of Vocational Education and Train- 
ing (VET) options in Australian secondary schools encouraged by 
Australia’s Ministerial Council on Employment, Education, Training and 
Youth Affairs (MCEETYA, 2004), there remains a struggle to achieve 
the 90% target for participation. The continuing issue for schools and 
governments is that for many students in their senior years, a senior 
school certificate and pedagogical approaches to student learning that 
do not accommodate these students’ broader interests and learning tem- 
peraments may not meet their needs. These are also themes that have 
emerged in the Going for Growth report released by the Organization for 
Economic Co-operation and Development: “The high share of the low 
skilled in unemployment and inactivity, especially early school leavers, 
suggests that the upper-secondary education system’s emphasis on prep- 
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aration for university is too narrow, hampering innovative activities and 
productivity growth” (OECD, 2007, p. 2). 

These issues were first identified in Victoria in 2000 by the 
Ministerial Review of Post Compulsory Education and Training Path- 
ways in Victoria (Kirby, 2000). This review indicated that a broader 
range of programs was required to meet the needs of the many young 
people who choose to leave school early because they were discouraged 
by the limited curriculum choices offered by the existing Victorian Cer- 
tificate of Education (VCE). The report also identified the need for new 
ways of thinking about senior school pedagogy and the social contexts 
of schooling in these years. The report states that: 


Those who have difficulties with current arrangements will typically 
need different learning contexts. (p. 10) 


Those young people who leave school before completing Year 11 
have experienced difficulties more often and have not adjusted well 
either to the VCE or to school as a social setting. (p. 53) 


In a context of increasing retention rates, many young people felt 
constrained by the current education system because their individual 
learning, personal development and transition needs could not be fully 
met by the traditional approaches to teaching and learning in the upper 
secondary years of schooling. As a result, many did not achieve their 
best results and others left school early, inadequately prepared for work 
and future learning. 


A DEMAND FOR APPLIED LEARNING 


The VCAL was therefore developed in Victoria as a state-wide 
senior school credential through which participating students re- 
ceive recognition for their achievements in programs that have tradi- 
tionally not provided credit within a senior secondary school quali- 
fication. Applied learning is central to the VCAL and students can 
achieve formal recognition of learning that occurs in a diverse range of 
non-traditional learning settings, including structured workplace learn- 
ing placements, part-time work, locally developed programs, communi- 
ty projects, and participation in youth development programs. Evidence 
of the strong demand for such an applied learning certificate being made 
available to senior school students is found in the very rapid growth 
of the VCAL since its initial trial in 2002, as summarized in Table 1. 

It is also worth noting that the VCAL is delivered by a range of dif- 
ferent providers, including: government, Catholic and Independent sec- 
ondary schools; Technical and Further Education (TAFE) Institutions; 
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and Adult and Community Education (ACE) organizations. In many 
cases a student’s VCAL may be the product of partnership arrangements 
between several of these organizations to allow students to achieve a 
combination of school-based studies and vocationally-based studies and 


Table 1: Growth in VCAL since 2002 (Victorian Curriculum and Assessment Authority, 2009) 


Year Enrollment Number of Providers 
2002 546 22 Pilots 

2003 5,127 239 

2004 8,125 322 

2005 10,692 380 

2006 12,326 401 

2007 14,093 418 

2008 15,641 423 


the eventual completion of all four curriculum strands of the VCAL. The 
overall curriculum design of the VCAL has been summarized in Figure 1. 

The Victorian Curriculum and Assessment Authority supports teach- 
ers’ use of applied learning in the VCAL by encouraging the context- 
based integration of theory and practice through a “hands-on” and real- 
life approach to learning and teaching. Key principles supporting this 
“applied learning” approach include: (1) Start where the learners are; (2) 
Negotiate the curriculum. Engage in a dialogue with the learners about 
their curriculum; (3) Share the knowledge. Recognize the knowledge 
learners bring to their learning environment; (4) Connect with communi- 
ties and real-life experiences; (5) Build resilience, self-confidence, self- 
worth — consider the whole person; (6) Integrate learning — the whole 
task and the whole person (in life we use a range of skills and knowl- 
edge. Learning should reflect the integration that occurs in real-life); (7) 
Promote diversity of learning styles and methods (everyone learns dif- 
ferently - accept that different learning styles require different learning/ 
teaching methods, but value experiential, practical and “hands-on” ways 
of learning); and (8) Assess appropriately. Use the assessment method 
that best “fits” the learning content and context (Victorian Curriculum 
and Assessment Authority, 2006). 


APPLIED LEARNING FOR RE-ENGAGEMENT 


Longitudinal research investigating the post-school destina- 
tions of young Victorians notes that “some 87.8 percent of VCAL 
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Figure 1: VCAL Design 


VCAL Program Design 


The Victorian Certificate of Applied Learning is accredited at three levels: 
= Victorian Certificate of Applied Learning — Foundation 
= Victorian Certificate of Applied Learning — Intermediate 
= Victorian Certificate of Applied Learning - Senior 


VCAL learning programs are comprised of four curriculum strands and teachers are encouraged 
to work with students to organize applied learning initiatives that will integrate across these 
strands. The four strands are: 
e Literacy and Numeracy Skills 
e¢ Work Related Skills 
Industry Specific Skills 
Personal Development Skills. 


Literacy and Numeracy Skills (LN) Strand: Purpose Statement 
The purpose of literacy curriculum selected for this strand is to enable the development of skills, 
knowledge and attitudes in literacy that allow progression in the main social contexts of family, 
employment, further learning and citizenship. 
The purpose of the numeracy curriculum is the ability to use mathematical skills in order to carry 
out purposes and functions within society related to designing, measuring, constructing, using 
graphical information, money, time and travel, and the underpinning skills and knowledge for 
further study in mathematics or related fields. 
Work Related Skills (WRS) Strand: Purpose Statement 
The purpose of the Work Related Skills strand is to develop employability skills and knowledge 
and attitudes valued within the community and work environments as a preparation for 
employment. 
Industry Specific Skills (ISS) Strand: Purpose Statement 
The purpose of the Industry Specific Skills strand is to enable the development of skills, 
knowledge and attitudes related to one or more vocational contexts in preparation for 
progression to further learning or employment. 
Personal Development Skills (PDS) Strand: Purpose Statement 
The purpose of the Personal Development Skills strand is to develop skills, knowledge and 
attitudes that lead toward: 

e social responsibility 

e — building community 

e civic responsibility, for example through volunteering and working for the benefit of 

others 
e improved self-confidence and self-esteem 
e valuing civic participation in a democratic society. 


students agreed or strongly agreed that the opportunity to enroll in 
VCAL was an important factor in their staying at school” (Teese, 
Clarke, & Polesel, 2007, p. 7). Students expressed a deep apprecia- 
tion for the experiential nature of the VCAL when asked to rate on a 
five-point scale what they liked most about the program (see Table 2). 

Many of the students undertaking the VCAL have experienced some 
level of disengagement from schooling and may have been considered at 
risk of leaving school early. Applied learning in the VCAL has therefore 
become a very significant factor in re-engaging these young people in 
schooling through its emphasis on a hands-on approach to learning and 
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a conception of education that goes beyond the more traditional school 
classroom. The extent to which this is the case is also reflected in the 
steadily increasing percentages of eligible students who successfully 
completed a VCAL certificate since 2004 (see Table 3). 

Finally, current national data indicate steady improvement in the 
Year 12 (or equivalent) completion rates, with an increase from 81.8% 
in 2000 to 88.7% in 2008 (in Victoria). These figures have been sourced 
from the Australian Bureau of Statistics (ABS, 2009) and summarized in 
Table 4. Australia’s average completion rate for all states is 84.2% (see 
Table 5). Victoria is therefore making progress towards the 90% target, 
and the VCAL has become a significant strategy for achieving this goal, 
along with a range of recently announced reforms that now guarantee a 
training place for all 15 to 19 year olds and 20 to 24 year olds who are 
up-skilling. 


Table 2: Students’ rating of different aspects of the VCAL program 


In relation to your VCAL program, which aspects Agree 
of your study did you like the most? 
(N= 1907) % Strongly agree % Agree % Total agree 

Workplace experience 48.0 43.9 91.8 

The opportunity to do Vocational Education and 

Training (VET) subjects 35-4 524 B78 

The hands-on approach to learning 56.8 39.4 96.2 
asin to complete your studies at your own 46.7 471 93.8 


Table 3: Completion rates by gender for VCAL units and completion rates for students 
eligible to receive a VCAL certificate 


. Satisfactory Total satisfactory Percentages of eligible 
Satisfactory aoa completion of 
Year : completion of 7 students who completed a 
completion of ‘ VCAL units : 
A VCAL units VCAL certificate at any 
VCAL units (males) (males and 
(females) level 
females) 
2004 No data available No data available 70.5% 58.7% 
2005 73.7% 72.2% 73.1% 63.8% 
2006 81.0% 79.7% 80.6% 68.7% 
2007 79.2% 71.7% 78.7% 69.4% 
2008 78.4% 78.5% 78.4% 70.0% 
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INTERNATIONAL GROWTH IN SCHOOL-BASED 
APPLIED LEARNING 


The significant potential for applied learning to underpin a senior 
high school qualification has also been recognized in other countries, in- 
cluding the United Kingdom, where the new applied learning “Diploma” 
qualification was recently introduced in schools (Qualifications and Cur- 
riculum Development Agency, 2009). This qualification also targets 14 
to 19 year old students and, like the VCAL, it represents a new vision 
in senior secondary schooling that adopts applied learning as the corner- 
stone of its pedagogical foundations. 


Motivation and achievement are increased when tasks have purposes 
that are relevant to the individual and ‘real-world’ contexts, includ- 
ing the workplace, and when learners are actively involved, and sup- 
ported, in shaping their learning experience...The collaborative nature 
of Diploma delivery is intended to secure a coherent experience for the 
learner that spans school, college, community and workplace environ- 
ments. The emphasis is not only on the development of knowledge, but 
on the ability to apply that knowledge purposefully to various contexts 
related to real-world situations such as workplaces. By participating in 
a ‘community of practice’ associated with a particular sector or sub- 
ject, learners gain experiences of different activities and groups, and 
develop the attitudes, as well as knowledge and skills, which are a pre- 
condition for progression. (Qualifications and Curriculum Authority, 
2008, pp. 2-4) 


Table 4: Percentage of persons aged 20-24 who have completed Year 12 or equivalent in 
Victoria, Australia 


Year 2000 2001 2002 2003 2004 2005 2006 2007 2008 


Percentage 
of persons 
aged 20-24 
who have | 81.8% | 82.1% | 82.8% | 84.9% | 84.7% | 83.9% | 85.5% | 86.1% | 88.7% 
completed 
Year 12 or 
equivalent 


Table 5: Percentage of persons aged 20-24 who have completed Year 12 or equivalent in 
2008 for all states and territories of Australia 


State/ New Victoria | Queensland South Western Tasmania Northern Australian Australia 
Territory South Australia | Australia Territory | Capital (average) 
Wales Territory 
Percentage 
of persons 
aged 20-24 
who have 


83.4% 88.7% 84.3% 82.1% 80.0% 72.0% 64.9% 90.4% 84.2% 
completed 


Year 12 or 
equivalent 
for 2008 
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The growing international interest in applied learning programs for 
senior high school students also extends to China and other countries in 
the Middle East and Southeast Asia. In the United Arab Emirates (UAE), 
for example, an applied learning program based on the VCAL has been 
introduced for senior high school students as a strategy to improve stu- 
dents’ engagement in learning and to address identified skill shortages 
anticipated in that region of the world. It is apparent that applied learning 
is increasingly valued in high schools for its intrinsic capacity to engage 
young people in authentic learning and meaningful social integration, 
while also contributing to a nation’s economic growth. 


CASE EXAMPLES OF APPLIED LEARNING IN THE VCAL 


The following cases provide examples of specific VCAL programs 
that have been undertaken by students in Victorian secondary schools. 
The cases have been selected because they demonstrate the use of ef- 
fective applied learning projects which have been used by students and 
teachers to complete the requirements of the different strands of the 
VCAL. They also demonstrate how far applied learning extends across a 
range of learning contexts in the community. 

Applied learning in the VCAL shares many of the pedagogical tra- 
ditions of experiential learning (Beard & Wilson, 2006; Dewey, 1938; 
Kolb, 1984; Luckner & Nadler, 1997) which have been blended with 
the principles of adult learning (Knowles, 1990; Mezirow, 1991) and 
adapted for secondary school contexts. The applied learning that is fun- 
damental to the VCAL program often requires students to assemble their 
VCAL from a variety of “hands-on” learning experiences derived from 
a diverse range of educational settings and community contexts. To com- 
plete applied learning projects VCAL students often cross organization- 
al boundaries that have remained historically difficult to bridge in the 
community. These boundaries include the traditional divisions between 
schools as youth-based learning institutions, adult and vocational educa- 
tion institutions, workplaces, and community organizations. 

In the case of John, for example, one of his applied learning projects 
involved the development of a Fire-Wise trailer to educate the local com- 
munity about the risks of wild fire. Bush fire is a very significant issue 
for many Australian communities threatened by fire every summer. This 
very meaningful aspect of his applied learning was reflected in his moti- 
vation to undertake the project. John’s applied learning project required 
him to complete traditionally adult-based Certificate II and I cours- 
es related to fire fighting, as well as working closely with his teachers 
from his school, the local Country Fire Authority (CFA) and the wider 
community. 
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Figure 2: VCAL Case 1, John and Country Fire Authority 


“John” is 16 years old and is studying to complete his VCAL in a small town in regional Victoria. In 
addition to his study he is a volunteer fire fighter in his local community with the Victorian Country 
Fire Authority (CFA). To complete his VCAL John is blending study at school with the applied 
learning included in his accredited wild-fire training offered by the CFA, and his work in the local 
community advising residents how to protect their home from a fire. 


His involvement in protecting the local community from bush fires not only makes a valuable 
contribution to the community but also allows him to complete his VCAL by drawing on meaningful 
applied learning activities linked to Australia’s National Training Packages. Examples of applied 
learning activities that John undertakes to assemble his VCAL from the different strands include: 
Certificate II in Fire-fighter Operations; Certificate II/IIl in Public Safety (Fire-fighting Operations); 
Certificates for Occupational Health and Safety; Level II First Aid Training; organizing media 
coverage and public presentations to the community; initiating, planning and implementing 
fundraising activities for a ‘’Fire-Wise” trailer to be used to educate the local community about 
managing the risks of bush fire. 


The combination of these applied learning activities allows John to complete key elements of his 
VCAL strands. Much of his learning undertaken for the normally adult-based Certificate II and III 
courses allow John to meet the Industry Specific Skills and Work Related Skills strands, while his 
community service role contributing to the education of the local community about bush fire 
contributes to his Personal Development Skills and Literacy and Numeracy Skills strands. 


Industry Specific Skills 
Strand 


Work Related Skills 
Strand 


Literacy and 
Numeracy Skills 
Strand 


Personal 
Development Skills 
Strand 


Adult-based certificate 
courses in fire-fighting 


Work-based learning 
in fire-fighting with the 
CFA 


Authentic publications 
developed for media 
and local community 
presentations 


Plan and conduct 
fund-raising activities 
for the development of 
‘Fire-Wise’ Trailer to 


be used to education 
the local community 
about bush fire. 


John’s applied learning project therefore required him to work across 
the traditional boundaries between secondary schools, adult learning 
environments, community service organizations (CFA) and workplaces. 
John’s experience of applied learning involved his personalized inte- 
gration of service-learning, work-based learning, and more traditional 
school-based learning as well. The flexible framework for the VCAL 
allowed John to actively construct and demonstrate his knowledge and 
skills related to each of the different strands by using his applied learning 
project. The transferable nature of his acquired knowledge and skills was 
also demonstrated by John’s ability to work competently in each of the 
different contexts that were used to assemble his applied learning proj- 
ect. The knowledge and skills that emerged from John’s applied learning 
project were truly multidisciplinary and had a very clear connection to 
his real world. 

The case of the River-Connect project also demonstrates the unique 
nature of applied learning in the VCAL. Like John, the 26 students from 
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Figure 3: VCAL Case 2, Notre Dame College and the River-Connect Project 


Notre Dame College, a Catholic secondary school in regional Victoria combined with the Goulburn 
Broken Catchment Management Authority, The Greater Shepparton City Council, Department of 
Primary Industries, Parks Victoria, Goulburn Murray Landcare Network, and Goulburn Valley 
Water to undertake an innovative conservation program known as the River-Connect project. 


The River-Connect project was part of a larger plan, developed by these local government and 
community agencies, to revitalize Reedy Swamp on the Goulburn River in the heart of the City of 
Shepparton. Twenty-six VCAL students and two teachers worked closely with the local 
government and community agencies involved in the project to develop an innovative VCAL 
Personal Development Skills unit. The project enabled the objectives of the River-Connect project 
and the outcomes for the VCAL students to be simultaneously met. 


The VCAL students worked closely with the government and community agencies to undertake 
systematic audits and surveys at Reedy Swamp. These were done using GPS technology and 
were used to assess the infestation of noxious weeds such as arrowhead and willows, the growth 
of red gum seedlings in the swamp and the giant rush that is used by water birds for breeding. 
The students were also involved in rubbish clean ups, poisoning willows and the planting of native 
trees in culturally significant areas of the sand dunes at Reedy Swamp. 


This program won a 2007 VCAL Achievement Award in the category of VCAL Partnerships. The 
school was also invited to present at the 9" International Partnership Conference: Interpreting the 
Present to Influence the Future in Helsinki, Finland. The VCAL program, involving the seven 
partners, received an honorable mention in the Global Awards presented at the conference. 


Industry Specific Work Related Skills Literacy and Personal 

Skills Strand Strand Numeracy Skill Strand | Development Skills 
Strand 

Adult-based VET Work-based learning | Researching and Restoration of Reedy 

certificate courses in conservation writing about the Swamp utilizing 

in volunteering and | activities (e.g., tree impact of noxious community 

conservation and planting, removal of weeds in Australia engagement and 

land management noxious weeds) (literacy) community service 


Presentation to 
community groups on 
activities to restore 
Reedy Swamp 
(literacy) 


Mapping of the 
distribution of noxious 
weeds in Reedy 
Swamp (numeracy) 


Notre Dame College who participated in the River-Connect project were 
required to assemble their applied learning by drawing on a meaning- 
ful project and a range of rich learning contexts that are rarely brought 
together for secondary school students. Applied learning for the River- 
Connect project involved students crossing the organizational bound- 
aries between their school, local government, environmental agencies, 
and the wider community. In the process students blended their school 
learning with service-learning, adult learning and work-based learning. 
The students’ service-learning contributed to meeting all of the learn- 
ing outcomes for the Personal Development Skills (PDS) strand of the 
VCAL, which was ultimately assessed through the students’ demonstra- 
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tion of them in an authentic context. The blending of the different types 
or modes of learning in the VCAL can be represented by Figure 4 below. 

A key feature of the applied learning essential to the VCAL is the use 
of “real-world” projects that allow students to become actively involved 
in constructing new knowledge and skills which are given meaning and 
purpose by the context of their application. This process usually takes 
the students beyond the school gates and, as demonstrated in the two 
case studies, applied learning projects in the VCAL require students to 
cross institutional boundaries and to personally assemble their learning 
from a range of different learning contexts. This feature of applied learn- 
ing in the VCAL means that students are also likely to be confronted by 
the differing assumptions about learning that are held in each of these 
settings. In workplaces, for example, the students are likely to experi- 
ence value being placed on practical forms of knowledge and skills that 
have very specific contexts of application. Fire-fighting and improving 
the environmental management of the local swamp are examples of such 
contexts from the case studies above. 


Figure 4: Blended Nature of Applied Learning in the VCAL 


School-based 
learning in 
classroom 

contexts 


Adult learning in 
vocational 
learning 
contexts 


Work-based 
learning in work 
place settings 


Project-based 
learning 
involving 'real- 
world’ issues 


Service learning 
incommunity 
contexts 


Such knowledge and skills are best acquired and demonstrated in the 
context of their application and less likely to be measured in an abstract 
setting. The students’ process of learning such knowledge and skills in- 
volves working with people whose core business is not education but 
who still have a significant investment in learning. Such people include 
the teams of fire fighters who worked with John, and whose community 
of minds and bodies represent the “library” of knowledge and skill to 
be learned by him. In this setting VCAL students are likely to discern 
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that the “best” forms of learning are highly situated and that “valuable” 
knowledge often resides in teams or communities of people. Assessment 
of the knowledge and skill acquired by the students is therefore associ- 
ated with the students’ contribution to the community and evidence of its 
impact on the community. 

Students are likely to experience different messages about what 
types of knowledge and skills are “most valuable” in secondary school 
settings (Blake, 2007). In this highly controlled educational context 
greater emphasis is placed on knowledge and skills that are generaliz- 
able, can be abstracted from the contexts of application, and then taught 
and measured in a classroom situation. Such knowledge is also likely to 
be located in books or the minds of educational experts. The process of 
learning in a traditional secondary school setting and associated with the 
completion of a more conventional senior school certificate, may there- 
fore typically be represented by the transmission of abstract information 
from the minds and books of teachers into the minds of students. In this 
situation students are likely to experience “valued” learning as being the 
acquisition and storage of generalizable forms of knowledge in their in- 
dividual minds. Assessment is usually focused on the student’s capacity 
to reproduce the knowledge under exam conditions. There is clearly the 
potential for VCAL students to receive conflicting messages about learn- 
ing and valuable forms of knowledge as they move between the different 
learning contexts for the VCAL. 


IMPLICATIONS FOR EDUCATING 
TEACHERS IN HIGHER EDUCATION 


The applied learning process in the VCAL requires of teachers new 
types of knowledge and skills that are responsive to the process by 
which their students cross boundaries to assemble their different modes 
of learning. Such pedagogical knowledge and skills exist in addition to 
teachers’ knowledge of their subject disciplines and share similarities 
with effective project management, including: (1) an ability to identify 
and link current community issues that are likely to be highly relevant 
to senior secondary school students and useful sources of meaningful 
applied learning projects; (2) an ability to develop and sustain creative 
partnerships between schools and other organizations in the wider com- 
munity who also have an interest in learning (such organizations include 
adult learning institutions, workplaces, and community service organiza- 
tions); (3) a broad conception of learning and deep appreciation of the 
different types of learning that students are likely to encounter as they 
become actively involved in their applied learning projects and cross 
institutional boundaries in the community; (4) an ability to assist stu- 
dents as they synchronize and assemble their own personalized experi- 
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ences of multi-disciplinary learning from each of the different settings 
involved in an applied learning project; (5) an ability to assess students’ 
knowledge, skills and competencies related to applied learning and assist 
students’ continual development in this regard; (6) a capacity to synchro- 
nize students’ assessment in applied learning projects by accessing and 
evaluating information about the students’ contributions to the learning 
contexts and communities beyond school; and (7) an ability to build ef- 
fective relationships with senior school students that reflect the princi- 
ples of adult learning and their status as young adults while maintaining 
the appropriate “duty of care” expected of a school teacher. 

Ultimately VCAL teachers are required to overcome many of the 
dichotomous divisions that have tended to dominate education institu- 
tions, resulting in the transmissive approaches that are often associated 
with senior schooling and the many reasons students give for wanting 
to leave school early. Applied learning projects in the VCAL require 
teachers to value learning that encourages greater links between: “learn- 
ing with hands and learning with the mind”; learning as individuals and 
learning as teams and communities; propositional knowledge that can be 
learned from books and tacit knowledge that is acquired through situated 
“hands-on” experiences. 


THE GRADUATE DIPLOMA OF EDUCATION 
(APPLIED LEARNING) 


In 2005 Deakin University established the Graduate Diploma of 
Education (Applied Learning) in response to the rapidly increasing de- 
mand for pedagogical knowledge and skills required to support the expan- 
sion of applied learning in secondary schools. The most immediate chal- 
lenge for the course developers was the use of applied learning in higher 
education to prepare teachers for applied learning in secondary schools. 
The immediate risk for course developers was that fiscal constraints in 
higher education would typically pressure delivery of a course on applied 
learning into the seemingly cost effective use of weekly lectures and tu- 
torials. This mode of delivery would have been epistemologically incon- 
sistent with the applied learning content of the course, which cannot all be 
reduced to a series of lectures. The course developers were, in fact, 
facing many of the same pedagogical challenges being faced by 
VCAL teachers. 

The course team developed a very strong relationship with the 
Geelong region’s Local Learning and Employment Network (LLEN), 
an organization funded by the Victorian State government to build ef- 
fective local community partnerships designed to better understand and 
improve pathways and transitions for young people in the region. There 
is a diverse range of community partners included as members of the 
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Table 6: Learning and Employment Organizations 


Learning and Employment Potential contribution for applied learning in 
Organizations teacher education 

Secondary Schools Contexts for the Graduate Diploma of Education 
(Government, Catholic, and Independent) (Applied Learning) teacher-education students to 


complete their own work-based learning and service- 
Technical and Further Education Institutions (TAFE) learning to better understand the current contexts in 
which young people are learning. 

Adult and Community Education (ACE) organizations | Source of different communities of professional 
practice to facilitate grounded development of 
students’ understanding about young people and 
applied learning in different “formal” learning 
contexts. 


Victorian Curriculum and Assessment Authority Important source of knowledge and information about 
current developments in applied learning in schools 

Department of Education, Employment and Early and other youth education settings. 

Childhood (DEECD) 


Provide important contexts for teacher education 
Local government and community organizations students to become involved in local community 
issues relevant to young people and to complete 
service-learning oriented towards youth engagement in 
learning. 

A context for teacher education students to better 
Employers and industry reference groups understand work-based learning in workplace settings, 
and consequently the significance of work-based 
learning for students in the VCAL. 


regional LLEN (see Table 6), meaning that the course team had access to 
many organizations within the community that would allow the develop- 
ment of substantial components of work-based and service-learning in 
the course. Much of the course development occurred from consultation 
between the Deakin University team and members of the LLEN, result- 
ing in a program that could be grounded in many of the important local 
issues affecting young people. 

The team decided that a “mixed-mode” approach to the course 
would be the best strategy to balance the need for face-to-face lectures 
and workshops with the applied learning dimensions of the Graduate 
Diploma of Education (Applied Learning). Mixed-mode is also de- 
scribed as a form of “blended learning” (Hicks, Reid, & George, 2001) 
that utilizes face-to-face teaching with other modes of learning, typically 
including on-line learning technologies. The Graduate Diploma of Edu- 
cation (Applied Learning) was developed using four different modes of 
learning that would provide the higher education students with a combi- 
nation of learning experiences and contexts that would also demonstrate 
congruence with applied learning in the VCAL. The four different modes 
of learning are: intensive face-to-face workshops delivered at the univer- 
sity and in school/industry-based contexts; on-line learning (off-cam- 
pus); work-based learning undertaken in schools, TAFEs or ACE; and 
service-learning contributing value to community-based initiatives that 
are oriented towards improving youth engagement in education and 
training (see Table 7). 
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Each of the five intensive teaching blocks in the Graduate Diploma 
of Education (Applied Learning) runs for two weeks duration and oc- 
cur as: Summer School; Autumn School; Winter School; Spring School; 
and a final one week block undertaken in November. The work-based 
learning mode in the program is completed as blocks of professional ex- 
perience placements undertaken in schools, TAFEs, or ACE settings and 
during the periods between the intensives, providing a total of 45 days 


Table 7: Summary of Mixed-Mode Learning in the Graduate Diploma of Education (Applied 
Learning) 


Mode of Graduate Diploma of Education (Applied Learning) 
Learning 
Face-to-face Five intensive on-campus teaching blocks conducted for two weeks at a time in February 


(Summer School), April (Autumn School), July (Winter School), September (Spring 
School) and November (Final block). 


On-line Students participate in on-line study using ‘Blackboard’ while they are “off-campus” in 
between study blocks and during their work-based learning placements 

Work-based Students complete 45 days of work-based learning under the supervision of a suitably 

Learning qualified mentor in secondary schools, TAFE institutions, and ACE settings. 

Service-learning Students undertake 15 days of negotiated service-learning requiring them to add value to 


their service-learning context. These days can be completed as small blocks of three and 
four days at a time or as a commitment to one more significant community 
initiative/project, depending on the nature of the activity. 


professional experience for the year. The 15 days of service-learning can 
be completed by students at any time throughout the course, with many 
students electing to complete most of their service-learning by support- 
ing initiatives and programs in the local community that benefit from 
their prior industry knowledge and skills. Finally, the on-line learning 
elements commence immediately after the first intensive is completed 
in February and continue to be undertaken by students in the periods 
between the intensive face-to-face study blocks. The on-line mode of 
delivery allows the students to continue to draw from and contribute to 
the course’s community of learners during the periods off-campus. 

The sequence of the eight curriculum units for the Graduate Diploma 
of Education (Applied Learning) is intended to cater to students’ devel- 
oping knowledge and skills as they progress through the course, as well 
as to synchronize their need to learn new content that will both support 
and utilize their work-based and service-learning projects. The students 
are encouraged to become actively involved in authentic school-based 
applied learning projects as key dimensions of their work-based learn- 
ing and service-learning. Although each of the eight units has its own 
discrete study design and “learning folder” on the Deakin Studies On- 
line (DSO) Blackboard web-site, the course team deliberately attempts 
to integrate the content and assessment of units as much as possible, 
particularly during the face-to-face teaching sessions. The intended aim 
of this strategy is to emphasize the students’ whole course experience 
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and to avoid the perception that any of the course units are overly dis- 
crete and unrelated to the others. 

In 2008 a pilot study (Blake & Nolan, 2008) was conducted to in- 
vestigate the higher education students’ response to blending the four 
different modes of learning being used in the Graduate Diploma of Edu- 
cation (Applied Learning) and the Bachelor of Early Childhood Educa- 
tion offered at Deakin University. In particular, significant numbers of 
the 41 students who participated in the research either agreed or strongly 
agreed (78%) that using the mixed-mode learning allows greater bal- 
ance between learning theory and practice. Additionally, a significant 
proportion of the students either agreed or strongly agreed (81%) that 
the knowledge and skills they learned through mixed-mode delivery was 
easily transferable between the different modes. 


CONTINUING CHALLENGES FOR THE VCAL AND 
PREPARATION OF “NEW” TEACHERS 


Many of the challenges facing the VCAL reflect challenges associ- 
ated with its rapid growth and success over the last eight years. These 
challenges also affect the delivery of the Graduate Diploma of Education 
(Applied Learning) and include: 


¢ The challenge to broaden many secondary teachers’ conception of 
learning and how learning can be assessed. In particular, the need 
to correct the view held in some secondary schools that applied 
and vocational learning is a “second class” way of learning only 
offered to “at-risk” students as a form of “therapy.” This view is 
inconsistent with the use of applied learning in higher education, 
where it is more likely to be valued as best practice (Wolff & Tin- 
ney, 2006) and a strategy to improve graduation rates. The “ap- 
plied-learning-as-therapy” view held by some secondary school 
teachers fails to recognize how more transmissive approaches to 
teaching contribute to many young people’s early departure from 
school (Blake, 2004, 2006). 


The challenge to meet the growing demand for applied learning 
teachers in response to the increasing demand for the VCAL to be 
delivered in the diverse range of youth learning locations, includ- 
ing secondary schools, TAFEs, and ACE organizations. Simulta- 
neously, the demand for these teachers is reflected in the need for 
experienced teachers who can add value to the work-based learn- 
ing and service-learning arrangements required by the pre-service 
teachers in the Graduate Diploma of Education (Applied Learning). 
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e The challenge to be creative in the synchronization of the differ- 
ent modes of applied learning essential to both the VCAL and the 
Graduate Diploma of Education (Applied Learning). This includes 
using highly effective applied learning projects that will encour- 
age the contextualized development of transferable knowledge 
and skills. 


e The challenge to equip teachers with the capacity to facilitate high- 
value applied learning partnerships between schools, universities, 
VET providers, employers, and members of the wider community. 


The challenge for the VCAL to be responsive to the rapidly con- 
tracting youth employment market created by the current econom- 
ic climate, and the need to diversify post-school study options for 
VCAL graduates. This also includes recognizing the potential for 
the VCAL to be a pathway of choice for students affected by social 
disadvantage as well as students who simply prefer applied learning. 


CONCLUSION 


The Victorian Certificate of Applied Learning was created in 
Victoria, Australia as a response to increasing participation rates in se- 
nior secondary schooling throughout the 1990s and early 2000s. The 
program is offered as a “hands-on” alternative to the more traditional 
Victorian Certificate of Education and has enjoyed strong enrollment 
growth since its introduction in 2002. It is now understood to play an 
important role in the achievement of youth education policy targeting 
90% completion of Year 12 or its equivalent. The VCAL has emerged 
in a policy context linking education to economic growth and a growing 
awareness of the personal, social, and economic risks created by young 
people leaving school early. 

Applied learning in the VCAL utilizes community-oriented proj- 
ects to achieve a unique blend of work-based learning, service-learn- 
ing, school-based learning, and adult learning. Students are encouraged 
to use their “hands-on” projects to personally assemble their learning 
across a diverse range of authentic learning contexts. This represents a 
significant shift in thinking for secondary schools, where transmissive 
approaches to teaching have tended to dominate the senior secondary 
curriculum since the earlier years of lower Year 12 completion rates. 

The VCAL has also created the need to re-think how applied learning 
can be used in higher education to better prepare pre-service teachers. 
This article has profiled the mixed-mode approach offered to students 
in Deakin University’s Graduate Diploma of Education (Applied Learn- 
ing). This model also emphasizes the use of applied learning projects 
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and attempts to achieve best practice by blending the use of intensive 
face-to-face workshops with elements of work-based learning, service- 
learning, and on-line learning undertaken when students are off-campus. 


GLOSSARY 


Adult and Community Education (ACE): A community based orga- 
nization that may be a registered training organization and/or an autho- 
rized VCE or VCAL provider. 

Department of Education Employment and Early Childhood 
(DEECD): A Victorian Government department responsible for learn- 
ing, development, health and well-being of all young Victorians from 
birth to adulthood. 

Local Learning and Employment Networks (LLEN): Community 
networks established across Victoria to support young people’s connec- 
tions with local education and training organizations, employers, and 
community groups. 

Middle years of schooling: Students who are in the fifth to ninth year 
of schooling. 

National Training Packages: National training packages are the foun- 
dation of Australia’s national training system. They are flexible train- 
ing products designed and developed by Australian industry to support 
training pathways in current and emerging vocational skill needs. All 
training packages incorporate national units of competency, assessment 
guidelines, and national qualifications. 

Post-compulsory years of schooling: Schooling that is completed after 
students have reached the minimum school leaving age. This age is typi- 
cally 16 to 17 in Australian states. 

Secondary schools: Schools that provide education from Year 7 to Year 
12. Secondary schools are also frequently referred to as high schools or 
secondary colleges. 

Senior school students: Students who are completing the last two years 
of their secondary schooling. These senior years are often referred to as 
Year 11 and Year 12. 

Strands: The VCAL contains four curriculum strands: literacy and nu- 
meracy skills, industry specific skills, work related skills and personal 
development skills. 

Structured workplace learning placement: Structured workplace 
learning is on-the-job training during which a student is expected to mas- 
ter a set of skills or competencies, related to a course accredited by the 
Victorian Registration and Qualifications Authority (VRQA). 

Technical and Further Education: Publicly funded and adult-oriented 
training organizations which provide a range of technical and vocational 
education and training courses and other programs. 
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Victorian Certificate of Applied learning (VCAL): An accredited 
senior secondary school qualification undertaken by students in Years 
11 and 12. 

Victorian Certificate of Education (VCE): The VCE is a senior sec- 
ondary qualification that is designed to be completed in Years 11 and 12. 
Victorian Curriculum and Assessment Authority (VCAA): A statu- 
tory authority that provides high quality curriculum and assessment pro- 
grams for Victorian students. 

Vocational Education and Training (VET): Nationally recognized 
vocational certificate. This certificate may be integrated within a VCE or 
VCAL program. 
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In the summer of 2008 a multi-disciplinary group of university students were provided an 
opportunity to tour on-site and observe facility staff in the course of their daily activities. 
Unlike the usual internship experience, the Juvenile Corrections Critical Assessment Tour 
allowed students access to nine juvenile facilities in four different states. Students were 
able to identify and evaluate personal preferences in system, facility, and mentor practices 
in a condensed four week period. Student-to-student and student-to-practitioner debrief- 
ings about specific observations in the facilities greatly enhanced the experience, creating 
a unique and effective workplace learning opportunity. By tightening the coupling between 
faculty in higher education and practitioners in the field, students were able to observe and 
explore lecture materials with the benefit of firsthand experience. 


The US Juvenile Justice System is not a system. It is a decentral- 
ized group of agencies bound by a vague composition of beliefs, but 
sharing the trend of being unsatisfied with the way things are—thus 
initiating on-going reform. Despite demographic similarities, even 
states who share borders practice an extremely diverse menagerie of 
legal responses to acts of youth. 
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So begins my initial lecture in Introduction to Juvenile Justice. Be- 
ing a former practitioner, I have the benefit of supplementing such dry 
lectures with firsthand experience, but I do limit this practice for several 
reasons. First, my experience is restricted to only a handful of states in 
the US. Second, students entering the vast juvenile justice arena could 
only be provided one aspect of the field: my experience in youth se- 
cure care. But more importantly, my experience is simply that—filtered 
through my own processing, bias, and even selective recall. A lecture 
hall does not allow the events and circumstances leading to decision- 
making to be fully demonstrated for students. This prevents students 
from challenging or processing the given claims. In addition, the foun- 
dation of the individual student’s knowledge or the wide array of student 
learning styles makes this task even more difficult for the instructor in 
a classroom. Providing students the opportunity to view firsthand the 
dilemmas they will face in their chosen field, while still maintaining a 
safe and effective learning environment, is indeed a daunting endeavor 
but extremely beneficial in creating the academic professional needed in 
such challenging fields as criminal justice. 


LEARNING THROUGH APPLIED LEARNING 


As an andragogical method, applied learning experiences in higher 
education are viewed as potentially effective means for students to in- 
terpret, process, and retain classroom instruction (Sims, 2006; Wolff & 
Tinney, 2006; Bailey, Hughes & Moore, 2004). Students who participate 
in these numerous types of learning activities tend to take less time to 
graduate and be more satisfied in the their vocation once they are em- 
ployed (Wimshurst & Allard, 2007). If done well, these courses can pro- 
vide students career exploration, increased skills and vocational social 
competence, as well as motivate students to complete the educational 
entrance requirements of their field (Hughes, Moore, & Bailey, 1999; 
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Bailey et al., 2004). Students who participate in these experiences seem 
to have a better understanding of what will be expected of them in their 
field of choice, eliminating much of the initial shock their new work- 
place will create (Sgroi & Ryniker, 2002). 

To prepare students to interact properly in the field, faculty must be 
aware of the details of the environment where the student will be placed 
(Hughes et al., 1999; Bailey et al., 2004). The communication lines be- 
tween student and practitioner should be made as level as possible if 
students are to learn from the environment they are placed in. “If done 
well,” these learning experiences can have the desired effect, but de- 
sign must be a primary consideration of the developer (Hughes et al., 
1999). Educators who make use of these types of learning experiences 
have heard much about the disjuncture between higher education and 
field experience. Traditional educational and workplace learning appear 
quite different at first glance. Education endorses and rewards individual 
problem solving, whereas this is rarely the case in a work environment, 
especially social services (Bailey et al., 2004). Primarily the necessity 
of building group or social relations to accomplish a given task is rarely 
rewarded in higher education. Higher education should focus its efforts 
on preparing people to be good adaptive social learners, so that they can 
perform effectively when situations are unpredictable and task demands 
change (Resnick, 1987). How to better couple both the workplace expe- 
rience and what the students take from their education during these field 
experiences may be a better focus of research and course development. 

A one-size-fits-all style of instruction does not suffice in the class- 
room and will limit the possibilities for student engagement in work- 
place-centered coursework. Learning styles differ for students as well 
as in disciplines. Using the Learning Style Inventory (LSI), Kolb and 
Kolb (1999) found students and even their vocational/educational area of 
choice to have identifiable tendencies. For instance, Kolb, Boyatzix and 
Mainemelis (2001) reported that students who favor the social service 
vocation were more apt to fall in a diverging style of learning, learn- 
ing best in a group or brainstorming type forum. Those scoring high 
in assimilating style were found to favor hard sciences and preferred 
individualized reading assignments and lecture. Students with converg- 
ing styles preferred simulations or experiments and were more likely to 
be in the fields of technology. Finally, accommodating style learners fa- 
vored hands-on experience and relied more on instinct than logic. These 
students were found in management or business vocations. Developing 
educational experiences which can juxtapose different styles of learn- 
ing can allow students to become active problem solvers, having built 
the ability to utilize different and sometimes uncomfortable methods to 
resolve tasks when in their field. 
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Allowing students in applied learning experiences to process alone 
what they view in the field is problematic. Educators tend to rely on the 
often misguided notion that all students will be able to recognize the use 
of theories or paradigms discussed in the classroom (Sgroi & Ryniker, 
2002; Bailey et al., 2004). Kolb and Kolb (2005) refer to a cycle of learn- 
ing, being a combination of “grasping and transforming experience” (p. 
194). Simplified, this cycle provides concrete examples, enriched by 
reflection given meaning by thinking and transformed by action. In es- 
sence, the cycle of learning described is not a typology but a process. 
Learning begins with the student identifying current beliefs or percep- 
tions made real by interactions with others. The synergistic transaction 
between the environment and the student beliefs or observation is put in 
motion when conversation occurs. In workplace learning this should oc- 
cur reflectively, both in the field and in the classroom. 


APPLIED LEARNING IN CRIMINAL JUSTICE EDUCATION 


Universities have actively pursued the mission to develop and market 
applied learning experiences for students (Bailey et al., 2004). Criminal 
justice departments have a long history of using internships and practica 
to allow students to see firsthand the diverse field of criminal justice. 
This is often dubbed “‘a real world experience’”—a somewhat concerning 
phrase—suggesting what higher education does in the classroom resem- 
bles nothing more than spouting unusable fiction to an audience. This is 
a very disturbing notion to one who remains passionate about the field 
of juvenile justice. Helping students process and develop as they experi- 
ence their chosen field should be the essence of a true applied learning 
experience, as well as a lecture hall. Maintaining a vigilant watch over 
the relevancy of facts provided these future professionals is a responsi- 
bility entrusted to instructors by students and by the practice. 

Criminal justice has received limited recognition in academics for 
the advancement of social science knowledge, theory, or methods with 
practice. The field struggles with legitimacy in the arena of higher educa- 
tion (Clear, 2001; Best, 2006; Finckenauer, 2005). An applied learning 
experience in the criminal justice field is not viewed as an academically 
sound experience equivalent to the caliber of residencies. This may be 
due to inaccurate perceptions by outsiders, but could also result as much 
from our own making in the discipline. We often fail to tightly couple 
education with the numerous workplace learning experiences our par- 
ticular discipline has to offer. 

Applied learning experiences in criminal justice programs are not 
typically represented by any one model. Common methods used to pro- 
vide the out-of-classroom experience a hint of academia involve person- 
al logs, journals, or summary papers (Stichman & Farkas, 2005; Bailey, 
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et al., 2004). Few of these classes attempt to prepare the student for their 
specific workplace learning experience. This may be due to the multi- 
tude of sites utilized by criminal justice academic departments, requiring 
unique preparation. The responsibility to prepare students to face what 
they will encounter in the field often falls by the way of “they will see 
it for themselves” (Stichman & Farkas, 2005, p. 148-149). In fact, the 
student’s and the department’s credibility with the agencies offering the 
applied learning experience hinges on how well the student is prepared 
to meet the needs of the agency. 


FIELD’S EYE VIEW 


Current practitioners seem to have a love-hate relationship regard- 
ing their involvement in higher education applied learning experiences 
(Shaefer, 1996; Biddinger-Gregg & Schrink, 1997). Those in the field 
who have experienced a tag along or shadow know the difficulties in 
balancing opportunities for a safe learning experience with simply hav- 
ing the intern do menial tasks that do not pose a risk. Few, if any, cases 
have been found where an agency has been found liable for injury to 
an intern student, but there exists in the back of any coordinator’s mind 
the possibility that situations may arise resulting in civil ramifications 
(Stichman & Farkas, 2005; Biddinger-Gregg & Schrink, 1997). 

The chance for potential employers to showcase what they have to 
offer the next generation of police officers, probation officers, and cus- 
tody staff, to name a few, is often cited as a reason for allowing intern- 
ships within an agency (Ross & Elechi, 2002). Agencies are also able to 
screen potential employees for a period of time at limited investment or 
risk responsibility, in the meantime providing a fresh audience to cur- 
rent employees (Bailey et al., 2004). Any experienced faculty member 
is easily able to recall a former student whose career jumping-off point 
was an internship experience, but details of how real and consistent these 
benefits are to the student or agency are limited. The vast majority of 
applied learning experiences that are allotted in the criminal justice field 
probably have more of a foundation in public relations and sense of duty, 
than recruitment of human resources. 


JUVENILE CORRECTIONS CRITICAL 
ASSESSMENT TOUR (JCCAT) 


In the summer of 2008, an applied learning experience was conducted 
with the assistance of juvenile facility managers in four different states. 
The Juvenile Corrections Critical Assessment Tour (JCCAT) course was 
designed to allow students the chance for guided discussion regarding 
one specific area of the juvenile justice system (juvenile corrections), 
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to tour facilities, and shadow veteran staff members on the job in nine 
different juvenile facilities in four states. A multidisciplinary group of 
students interested in working with delinquent youth were recruited for 
this course. Quite different than the usual course offering, JCCAT ended 
up being dubbed the ‘Shawshank Tour’ by numerous administrators and 
supporters at the university. 

The initial perception that JCCAT would simply provide students 
tours of facilities was not completely unexpected. In many ways the goal 
of the course was similar to the goal of the numerous tours offered dur- 
ing any academic year in criminal justice: to allow students the opportu- 
nity to view the inside of a facility. Unlike the simple tour, however, this 
course also allowed students to comparatively evaluate key differences 
in state care of adjudicated (sentenced) youth. Missouri Western State 
University provides a unique geographic location (northwest Missouri) 
for students to evaluate judicial, administrative, and program differences 
in the care of youth in four different states. Usually this is left for the 
new professional to learn only after accepting a job, which can lead to 
disenchantment with the field for the graduate, who may not realize a 
different paradigm is simply across a state line, a few miles away. 

The second goal of the JCCAT course was to allow these state fa- 
cilities the opportunity to “showcase” their program efforts for youth 
and attract quality employees who are interested in working with youth 
within their particular area of expertise. Unlike a simple tour, the 
JCCAT allowed facilities to do this within academic disciplines or by 
staff functions. After discussion with facility managers it was deemed 
best that a wide range of educational disciplines should be recruited, 
despite the course being offered through the Criminal Justice and 
Legal Studies Department. An institutional environment requires many 
services and is essentially its own self-contained community needing 
numerous types of professionals to operate successfully. Facility manag- 
ers played an active role in the development and orchestration of JCCAT. 
The facility managers and state administrators approached in these four 
states were unanimously in favor of being a part of this course. Their 
involvement did not simply end at providing access; these youth care 
professionals assisted in the development of the class, adjusting facility 
scheduling, while some even provided funding and meals for the class. 
Most importantly, these facilities gave 3-5 hours of their day in each of 
the facilities visited. 

Recruitment for the class occurred in the early spring of 2008. Due 
to class budget constraints and maintaining a focus on facility security, 
the class was limited to 12 students. After the initial interviews were 
conducted, two simply did not attend any further preparatory meet- 
ings and two dropped for personal reasons just prior (one week) to the 
course beginning. By design, no deposits or coercive means were used to 
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guarantee a set number of participants. Willingness was viewed as a 
necessity for this project to be effective. The eight remaining students 
were from four different disciplines and two were undecided, with 
the majority (4) being from the Criminal Justice and Legal Studies 
Department. All students were at least sophomore year status and one 
student attended a different college than the others. 

The course was designed to model a workplace applied learning 
experience for students, incorporating a strong emphasis on reflective 
models of student learning. This would allow structured lecture content 
to fill the necessary background students should have prior to their first 
field experience in juvenile corrections, while still supporting contextual 
exploration of the experiences. Several of these students were from dis- 
ciplines outside criminal justice, so it was imperative a general under- 
standing of juvenile justice was provided. Techniques of programmatic 
review (audit) were provided students in the classroom as well. A full 
week of intensive classroom instruction was provided on lecture topics 
such as: history of youth care in the United States, current status of youth 
care, risk/needs assessments, ethics, treatment modalities, and on-the- 
job stress. In addition, interview skills, field-specific terminology, and 
documentation training were also added to increase the perceived cred- 
ibility of these students to practitioners. Students were divided between 
the four states (two per state) and responsible for developing a state 
history of youth care and an organizational description of the state’s 
youth services. 

Prior to the class beginning, instructors predicted that the numer- 
ous tours would run together for students and faculty. Having conducted 
interviews of staff on site, instructors knew it was imperative for students 
to capture their initial thoughts and feelings regarding their mentor and 
the facility prior to moving on to the next site. Impromptu conversational 
transactions regarding facility occurrences would surely be forgotten or 
mistaken by the end of the course. To remedy this, two instruments were 
developed in the class during the first week of lecture and tour prepara- 
tion. The first was referred to as “Describe Your Mentor.” This instru- 
ment was to be filled out immediately after the student left the facility. 
Each JCCAT participant was asked to provide the general demograph- 
ics and tenure information about the staff serving as their mentor. In 
addition, descriptive characteristics regarding their guide’s personal- 
ity, how they treated the student (as a peer, mentee, know-nothing), and 
what they did with their mentor during their time were documented. In 
developing this tool the class was guided away from a ‘check sheet’ 
format. They were encouraged to use their own descriptive abilities in 
their documentation of the experience. The class settled on a tool and 
made only minor adjustments after the first JCCAT experience. 
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A second instrument, “Describe Your Facility,” was designed during 
class lecture preparation as well. One of the two student representatives 
of the state the facility was located in would interview their classmates 
immediately after a JCCAT site. Because travel took place after each 
tour, students were usually interviewed en route to the next destina- 
tion—creating a 70 mph captive environment. Instructors driving the 
travel vans were cognizant of peer impact, so no third party interruptions 
were allowed until all interviews were completed. Student interview- 
ers asked their fellow JCCAT students to identify what they observed 
regarding facility culture, to describe the youth they observed, as well as 
their opinion of facility security level (low, medium, high). One of the 
more telling questions in this instrument was simply “Would you work 
here?” A simple yes or no to this question was not accepted. Interview- 
ers were encouraged to probe for reasoning behind the student’s conclu- 
sion. Finally, the two state interviewers also interviewed each other and 
summarized their findings in their final project, which merged the state’s 
organizational mission and history with the JCCAT experience. 

In the second week of the summer class, students had their first JC- 
CAT at a state juvenile male facility. As designed, students were oriented 
by a facility administrator prior to being given about 2.5 hours to spend 
with front-line staff members during the course of their duties. Students 
were paired with group leaders, youth care workers, therapeutic profes- 
sionals and some administrators, depending on each student’s interest 
area. Course instructors were allowed to observe interactions, but at least 
one instructor remained in a central location for contact if necessary. 
Debriefing took place after the facility visit with facility administrators 
and the following day in the classroom, as a class. The first mentor sum- 
maries and student interviews were conducted with the assistance of 
the co-instructors for the course, and minor adjustments were made to 
the instruments. 

Administrators hosting the first JCCAT followed the desired course 
blueprint precisely. This allowed JCCAT students to have a better un- 
derstanding of expectations prior to embarking on a six day, seven facil- 
ity whirlwind tour of Iowa, Kansas, and Nebraska juvenile correctional 
facilities, finishing their JCCAT in Missouri during the final week of the 
class (see Figure 1). The class visited five all-male, two all-female, and 
two co-ed correctional facilities during the JCCAT tour. Again, written 
summaries and group debriefings took place after each facility visit. Two 
of these facilities were mechanically secure with a fenced perimeter, but 
the majority of facilities resembled more of an open campus—often with 
only locked living quarters between the residents and off grounds. As 
stated by one JCCAT student: “Some of these looked like high school 
campuses, not correctional facilities.” 
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Figure 1: JCCAT Tour Map 
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KEY: In order of JCCAT: 1. St. Joseph, MO (male); 2. Toledo, IA (co-ed); 3. Eldora, IA (male); 4. Atchison, KS 
(male); 5. Larned, KS (male); 6. Beloit, KS (female); 7. Kearney, NE (male); 8. Geneva, NE (female); 9. Watkins 
Mill, MO (co-ed). MWSU-Missouri Western State University, St. Joseph, MO. 


The JCCAT class logged over 1600 miles, visited nine facilities in 
four states, and experienced a truly unique and intensive learning experi- 
ence in a four-week period. The efforts of the facility staff involved in 
these tours fell consistently above expectations of instructors and stu- 
dents. Open praise for the students and the JCCAT project was received 
from facility and staff both during and after the course. In general, all 
facilities followed the course blueprint with only minor alterations. Ac- 
cess to facility staff was limited at one facility, but program review, ad- 
ministrator Question and Answer (Q&A), as well as on-site lunch with 
youth allowed adequate facility review for the students. The majority of 
facilities even allowed JCCAT students access to youth during mealtime, 
in Q&A sessions, and several times in institutional hearings. According 
to student responses, it was these contacts with youth that trumped all 
other aspects of the tours. 


EVALUATING THE EXPERIENCE 


How to evaluate and measure the academic preparation for this un- 
usual learning experience was a concern in its design. After the tours 
were completed, during the last two days of class, activities and discus- 
sion summarized the overall experience. This included open discussion 
of state systems and individual facilities (and their occupants). Students 
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were allowed to use their interview sheets to assist in the recall of spe- 
cific facility and staff. A final exam or post-test was provided at the end 
of the last day of class. Test responses were not factored into the course 
grade and students were again allowed to use documents to assist them 
in recall when needed. The final test consisted of scenario questions 
such as: 


Your potential employer’s interview committee sees Law 396 
JCCAT on your list of experiences that are job-related. They ask 
you how completing a college class would make you a more quali- 
fied employee in the juvenile justice system. What would you say? 


Over half of the students used the phrase “hands on” directly in their 
response to this question. One student wrote: ”I got to participate in 
everyday happenings...I didn’t just get time with staff...got time with 
youth: boys AND girls.” Almost all used phrases such as: “behind the 
scenes,” “behind the walls,” or “what really happens” to describe their 
JCCAT experience, also focusing on what can be learned outside the 
class and “textbooks.” But most impressive to the instructors of the 
course was discussion of the differences in programs that were identified 
in these states. Four students directly applied their response to career 
goals, such as: “I could see for myself if it was the course I want my 
career path to take,” and “I am well aware of the time [and]....additional 
training that will be required of me.” 

Based on a review of the student responses, it is apparent the learn- 
ing experience outside the classroom is what stood out most to these 
students. Not surprisingly, lecture or in-class presentations by instructors 
were not as memorable in the response to this question. The work with 
professionals in the field as well as time with youth was the focus of 
their responses. 

A second scenario question changed the audience for the students, 
asking them to identify to a graduate school advisor what JCCAT was 
and was not: 


Your Graduate School Advisor sees Law 396 JCCAT on your tran- 
script and is unsure what it should count towards. Your advisor asks 
you to describe the class. What do you say? 


JCCAT students framed their responses more to the practicum/in- 
ternship experience, focusing on either differences in state systems, or 
state philosophies of rehabilitation and security. One student stated: 
“We discovered how MO, KS, NE & JA differed in their facilities and 
their course of how the juveniles were released”—focusing more on the 
systems aspect, while another focused on her direct experience with 
“staff in the psychology-related fields,” which was her academic field 
of choice. 


Lindsteadt, Williams-Decker / LECTURE TO FIELD 83 


Students were also asked if they believed they were properly pre- 
pared for the tour component of the class. Overall, students indicated 
they felt properly prepared and all indicated it was a very helpful course 
for those interested in this field. Two students indicated they did not feel 
properly prepared, but in a follow-up interview with these students, one 
stated: “It was more of a shock and awe than what could have been 
done in lecture.” The second student told instructors: “I don’t think I was 
mentally prepared for the facilities....[ don’t think that I was unprepared 
educationally as to what to expect, but it was a pretty draining experi- 
ence. Definitely worth the work, though.” The lasting effect this work- 
place learning experience had on these students was encouraging to the 
instructors, to say the least. 

A third student brought up an interesting point regarding the col- 
legiality of the class and stated it would have helped the tour to dis- 
cuss this more often. The instructor’s role in building the necessary 
esprit de corps in a class depends greatly on the makeup of the class and 
class ownership of the task at hand. Faculty were consulted initially by 
students about other students who did not follow schedules or lacked 
input. By the end of the class there was a distinguishable divide, but 
students maintained a civil response that did not interfere with project 
goals. Students were asked to self-rate their collegiality in the JCCAT 
post-test. All self-rated themselves as fair to good, but no one identified 
poor or excellent. Interestingly, “classmate collegiality” was rated over- 
all excellent by half the attendees and only fair by one. 

Post-tour interview sheets were copied, collected and not allowed to 
be altered by students. The design of these tools was to capture the initial 
impact of the tour site. Several students voiced their concern about this 
during the process, due to their ability to “compare apples to apples” 
(student comment) as they saw more facilities. The final classroom dis- 
cussion allowed students to review all of these facilities and the state 
systems in hindsight in preparation for the final exam given at the end 
of the last day. The question “where you would like most to be em- 
ployed” was again asked in their final exam. Students were asked to 
rank state systems and facilities. The facilities were also divided by gen- 
der to evaluate preferences by students. Two state systems were ranked 
either 1 or 2 (highest) by all except one student. This single exception 
was a student focused more on a custodial approach than the majority of 
other students. Interestingly, when identifying which facility the students 
would choose to work in (male, female, all), these individual facilities 
did not always fall into the same state system the students chose as their 
top two. One such facility (female) appeared as a particular favorite for 
JCCAT participants despite the state it operated in not being ranked e1- 
ther 1 or 2 by any student. Comments offered compassion for that partic- 
ular facility’s plight: “Could use/needed my help” and “Did much more 
with less...compared to the boys facility.” 
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The final exam also requested student input on “Who should deter- 
mine how long a youth should be committed to a facility?” This ques- 
tion was designed to allow JCCAT students the opportunity to merge 
their understanding of a system’s larger structure with those required 
to carry out the tasks of the job. Two states in the JCCAT made use 
of an indeterminate sentencing scheme, where the facility staff was re- 
sponsible to determine how long a youth was committed. The other two 
states made use of a more formal determinate sentencing scheme, with 
judicial and administrative input into sentence length. This topic was 
discussed in class lecture, but it was uncertain if students were able to 
see the difference in release discretion at the facility where a youth was 
committed until discussing the benefits/dangers with staff mentors and 
administrators from different states. Course instructors predicted that all 
students would support facility-based control of release, but for different 
reasons due to the student’s academic discipline. Instructors believed 
that the criminal justice students would support control of release due 
to the behavior management benefits it provides, while the psychology/ 
sociology/social work discipline would support facility based release for 
the individualized approach to rehabilitative care. 

Directly contrary to the predicted results, no criminal justice JCCAT 
students supported release coming from facility staff where the youth 
was committed. We expected that discretion would be considered very 
important to future professionals, but apparently a justice approach was 
considered more desirable by this criminal justice group of students. 
Only JCCAT students coming from the psychology/social work perspec- 
tive were unanimously in favor of facility-based release, focusing more 
on an individualized approach to release readiness. 

To assist future offerings of JCCAT or other such multi-site applied 
learning experiences, students were also asked to rank travel, hotels, per- 
sonal time allotted, and facility preparedness. The services encountered 
in these mostly rural communities where facilities are located were found 
to be extremely hospitable and accommodating to the limited budget of 
the trip. All students ranked meals, hotels, and travel comfort “above 
average.” Only one student ranked “facility preparedness” as fair, but all 
other aspects of JCCAT organization and “comfort” were ranked “good” 
or “excellent.” Classroom climate in experiences such as JCCAT does 
involve much more than the typical collegiate lecture hall. The learning 
environment should not be too strenuous, and concern for “down time” 
for these students to digest this experience was considered very neces- 
sary. One student directly commented on her fear that “overnight stays 
sounds like you may never have free time, but as soon as you’re out of 
the facility [and] conversed [with other people], you got time to yourself.” 

Students were asked in their initial interview (prior to the course) 
if they had any apprehensions about entering a secure facility. JCCAT 
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was, in fact, several students’ first experience inside a secure facility. In 
the final exam, all students identified feelings of safety being “excellent” 
during the JCCAT experience. This is certainly a testament to the facility 
mentors and managers who were involved in this course, but one that 
should not be taken for granted or assumed when developing workplace- 
designated applied learning experiences. Specific attention should be 
given to how students view their learning environment to ensure there 
are no roadblocks, real or perceived, to this experience. Reflective, on- 
going communication between instructors and learning groups can assist 
in achieving this goal. 


LECTURE CONTENT 


To provide JCCAT students credibility with the facility staff, as well 
as the ability to evaluate key components of facility culture, it was im- 
perative the class was properly prepared. In an analysis of the first week’s 
lecture content, students were asked to identify lecture topics they actu- 
ally heard being discussed in the facilities. Once again, students were 
allowed to consult their facility notes. The students identified each of 
the lecture topics discussed in class preparation as being addressed by 
staff or administration in the facilities visited (see Table 1). Interestingly, 
every student reported that Job Stress was discussed by staff at every 
facility they visited. Other topics reported as being discussed in the ma- 
jority of facilities (median of 3 or higher: over half but not all facili- 
ties) were: Outside Regulating Agencies, Risk and Needs Assessments, 
Juvenile System, Responsivity, Sentencing, Outcome Measures, Dis- 
cretion, Difficulties of Reform, Treatment Modality, Rehabilitation, 
and Culture. 

Due to the relatively small class size, limited generalizations can 
be made. What is important to faculty preparing to teach workplace- 
centered courses is to be able to identify topics that are also significant 
to employers/employees where the students are going to be placed (in 
the case of JCCAT, juvenile correctional facility staff). This will ensure 
coursework is geared to current practice, allowing students to be in tune 
with what is of most concern in their desired occupation. For example, 
based on these interactions between student and facility staff, job stress 
is a very real issue to those working in the field. Students were able to 
discuss job stress that veteran and newer staff experience on a daily basis 
and how they get through these situations to show up another day. As 
discussed in lecture, students found that it is rarely the youth that are in 
the staff member’s care that create the most stress on the job for staff. 
It is instead their peripheral responsibilities and roadblocks to what they 
view is necessary to help the youth. 
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Table 1: Students’ recall of lecture topics discussed in facility: N=7. Median is reported (right 
column). 


Regarding lecture content: Please circle the 1 2 3 4 Median 
appropriate answer to the following lecture 
topics you heard being discussed or Not Some Over All 
mentioned while in the facilities. discussed _ facilities half facilities 
in any but not but not this was 
facility over all discussed 


half facilities 


Outside regulating agencies 0 1 4 2 3 
Ethical dilemmas 1 3 3 0 2 
Risk and needs assessment 0 0 2 5 4 
Juvenile corrections history 0 4 2 1 2 
Juvenile system as it relates to juvenile 0 2 3 2 3 
corrections 

Females in juvenile corrections 0 4 3 0 2 


(difficulties, needs & response) 


Responsivity 0 1 5 1 3 
Sentencing (as it relates to facility goals & 0 1 1 5 4 
practices) 

Outcome measures 0 3 2 2 3 
Discretion 1 2 4 0 3 
Disproportionate Minority Confinement 0 5 2 0 2 
Difficulties of reform 1 2 3 1 3 
Treatment modality 0 1 3 3 3 
Rehabilitation 0 1 4 2 3 
Juvenile crime rates and victimization 0 0 5 2 3 
Job stress 0 0 0 7 4 
Culture 0 2 4 1 3 


Somewhat concerning in these findings was the limited discus- 
sion of disproportionate minority confinement by those in the facility. 
This issue remains central to introductory coursework in juvenile jus- 
tice, but appears to be considered less pertinent by facility staff. A pos- 
sible explanation considered during debriefing was it may be the staff 
member’s willingness to work with whoever is committed that makes 
the difference, taking little responsibility for how the youth arrived in 
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their care. In addition, the fact that the racial makeup of these facili- 
ties has changed little over much of the tenure of the staff could lead to 
the application of “that is the way it is, has been, and will be” to justify 
these trends. JCCAT students quickly identified differences in staff racial 
makeup relative to the population of youth they served. Interestingly, 
this point seemed to lose its initial effect on students as more tour sites 
were visited—desensitization that is not so different than what is experi- 
enced in the field or discipline. 

Based on student reflection during the final exam, the lecture prepa- 
ration was overall on topic to the current concerns of those in the juve- 
nile correctional field. All topics discussed in the classroom were identi- 
fied in at least one facility by students. Notable to instructors was the 
students’ understanding of terms such as Responsivity or Culture, which 
would become evident during debriefing sessions. These are not always 
easy concepts for the student, but when the student could apply each idea 
to active interactions with staff and facility observations, they demon- 
strated the ability to translate the meaning of the academic term into the 
practical events and common facility language. Workplace learning may 
not always appear as a direct reflection or in the context of a specific ex- 
ample used in the classroom. The student’s ability to translate and utilize 
these concepts is indeed a proud moment for their instructors. 


DISCUSSION 


Allowing students to view and process the difference in state missions 
and juvenile facilities was the focus of this applied learning experience. 
To teach how an organization’s mission is reflected in the performance 
of an individual facility is a point difficult to drive home to students 
via textbook and lecture hall. Even if the student is fortunate enough 
to have field experience, this is usually limited to one system. In addi- 
tion, the ability to compare like experiences in the field with peers and 
faculty enhanced the learning process. This often occurred via agitation 
within the group of students. For example, due to the multidisciplinary 
makeup of the class, some viewed with disfavor the open campus of fa- 
cilities while others challenged the razor wire surrounding other facility 
grounds. This provided some interesting exchanges within and outside 
the classroom, which would occasionally require refereeing. Students 
were encouraged to look past their initial opinions and impressions, but 
not to ignore them. To provide a true critical assessment of a facility’s 
mission, students were challenged to examine what are the goals of the 
organization first, before they determined whether the facility is meet- 
ing those goals. As seen firsthand by the class, a system that focuses on 
custody and professionalization will appear different than one that has 
a dynamic focus on peer involvement in treatment. A student may feel 
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more comfortable working in one type of system or the other, but this is 
not to say that the other is performing its mission poorly. 

Few issues raised more vocal concern with the JCCAT class than the 
differences between male and female facilities. The female youth facili- 
ties were viewed by the class as insufficiently funded and in poor physi- 
cal condition. In an interview by our university magazine, one JCCAT 
student stated: “I was upset and shocked. ...It was space age for boys 
and pioneer for the girls” (Holtz, 2009, p.6). The class debriefing of this 
issue provided an opportunity to examine a state’s willingness to fund a 
philosophy of community protection versus individual youth need. The 
perceived social threat of the young male delinquent versus the need to 
protect the female delinquent was discussed, as it is in most Introduction 
to Juvenile Justice/Delinquency textbooks. Unlike traditional lectures, 
the class was able to use personal observations as reference to this dilem- 
ma. For several JCCAT students, these facility funding differences led to 
a change toward commitment to work with girls if given the opportunity, 
even despite the vast majority of staff in male, female, and coed facilities 
identifying delinquent girls as being more “difficult” and “challenging” 
to work with than male youth. 


DEVELOPING LEARNING SPACE 


This paper would be incomplete without addressing the necessity of 
timely debriefing and creating the necessary learning spaces in work- 
place experiences such as the JCCAT (Kolb & Kolb, 2005; Bailey et al., 
2004). Issues such as facility staff demeanor, youth behavior, and some- 
what challenging ethical decision making by staff were considered and 
reviewed within a short period of time between students, students and 
faculty, and also with practitioners. Students were instructed to remain 
focused on their task, but to discuss these incidents with course instruc- 
tors who would then present the incident to facility administrators. Three 
such incidents did occur during JCCAT. 

The benefit of immediate processing of student observations and 
concerns was evident in the confidence the class displayed in their in- 
teractions with facility staff and each other as the JCCAT progressed. 
Students even began requesting additional debriefing sessions to discuss 
their observations, indicating a personal commitment to the project. This 
was also evident in the final exam comments: “more time dedicated to 
interviews” and “group meetings (class, not institutional) were good but 
should be used more often.’’ Courses that apply active field participation 
for the future professional should consider the position of these students. 
The student is left to disentangle ethically challenging experiences with- 
out assistance from faculty or peers. In addition, relying solely on the 
practitioner’s perception of how daily activities merge into the overall 
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organizational mission will most likely limit the growth of the student’s 
understanding of their future vocation (Sgroi & Ryniker, 2002).This can 
result in the student developing an overall negative perception of the 
agency and field, when in fact it may be simply an isolated experience 
(Jordan, Burns, Bedard, & Barringer, 2007). 


JCCAT SUMMARY 


In hindsight, this class was an exhausting endeavor for instructors 
and students. These types of experiences abound with the possibility of 
pitfalls in planning, student (faculty) personality clashes, and facility in- 
cidents threatening the advancement of the project. In this case, these 
types of problems failed to rear their ugly heads, making the result fully 
worth the effort. It was refreshing to see the excited undergraduate stu- 
dent knowledgably discussing issues with professionals in the field. Stu- 
dents intrigued by what they hear, view in the media, or read often have 
an almost uncontrollable desire to be a part of something they have only 
observed haphazardly. By coupling knowledge of the system with actual 
observation, these JCCAT students were able to develop an informed 
outlook of the impact the individual facility staff can have on achieving 
an organization’s mission. This is especially important for students en- 
tering a field where subsystems exist within a system, but each maintains 
a diverse and sometimes competing group of activities. For example, the 
juvenile justice system has judges, police, and correctional workers all 
participating in a general mission, but having specific duties that tend 
to overlap and conflict (Leiber, Schwarze, Mack, & Farnworth, 2002). 
Teaching early that efforts of cooperation do matter as a way to reach the 
mission of an organization is a necessity for educators. 

The JCCAT class should be commended for improving the relation- 
ship between academia and current practitioners in this specific field. 
One facility manager wrote: “This was a great day...many of the staff 
involved appreciated being able to ‘show off’ their talents and educate 
others. It was a win/win situation for everyone.” The accolades given 
students by facility representatives were numerous and seemed to fo- 
cus clearly on the level of understanding of the JCCAT class of system 
mechanics, or “how prepared they were for what we do” and “the feed- 
back I received from my staff is that your students were great.. They 
seemed real interested and asked good questions. My people enjoyed 
your people, come back ANY time.” The post-tour Q&A with adminis- 
trators would often resemble a peer discussion more than a classroom or 
training session. By the end of the JCCAT tours, facility staff and the stu- 
dents were openly comparing program and systems from different sites 
where these practitioners had also traveled. These experiences serve as 
confirmation to the value employers place on those with college degrees, 
easing the transition from student to practitioner. 
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Although the co-instructors were happy with the student turnout and 
efforts for this course’s inaugural run, future JCCAT type courses could 
be greatly enhanced by expanded recruitment and developing methods to 
ease the burden for students to participate. These methods could include 
student travel funding, scholarship opportunities, or simply making 
space available in curriculum requirements to allow students the chance 
to personalize their own education. In this class, having male students 
involved may have provided a significantly different perspective to the 
information gleaned from those on the tour. Facility administrators also 
commented openly on the need for younger male role models for the 
youth in their care, a pool higher education can provide these facilities 
which are often found in remote, rural areas. 

Applied learning experiences in the workplace provide faculty a 
unique way for students to remain bonded to their field of interest. A 
thorough understanding of organizational needs will greatly increase 
the course planner’s ability to develop these types of applied learning 
experiences (Jeffords, 2007). Early in the course planning, facility ad- 
ministrators discussed the reality of one academic discipline not being 
sufficient to produce an effective environment to change troubled youth. 
In retrospect, the tension produced in mixing academic disciplines fur- 
ther enhanced the learning experience and, in the end, served to better 
prepare these future professionals for this challenging vocation and the 
wide range of individuals they will be working with. Academics must be 
willing to cross the often well guarded academic boundaries to develop 
the types of workplace opportunities that will benefit both the student 
and the field. 
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Student motivation is a universal teaching challenge. A holistic approach to assessment 
was developed for cognitive and psychomotor tasks in equine studies. First-year students 
(n=55) were either randomly provided (PR) or not provided (NP) a rubric 3 weeks prior to 
skills testing. The PR students earned lower total scores (p < 0.05) than NP students (12.8 
and 17.1 +5.3, respectively). In individual categories, PR and NP students had similar (p 
> 0.05) pass superior scores. Third-year students (n = 7) self-rated task performance us- 
ing an affective rubric and reflection exercises. Although the original goal was to promote 
standardization of hands-on skills, these data indicate that students are more goal-oriented 
than process-oriented; furthermore, use of affective rubrics for self-assessment promoted a 
learner-centered approach to motivation. 


Midway College held a virtual monopoly on equine programs in 
Kentucky when its program was started over 20 years ago. As student 
and industry demand for equine academics has increased, approximately 
185 institutions in the United States have launched programs with di- 
verse offerings as degrees, concentrations, or coursework focusing on 
the horse industry (National Association of Equine Affiliated Academ- 
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ics, 2009). This demand has followed the growth of the equine industry 
in the United States. The American Horse Council (2005) reported in a 
national study targeted at horse owners that there are approximately 9.2 
million horses, which is higher (3.6 million) than the untargeted study by 
the United States Department of Agriculture, National Agriculture Sta- 
tistics Service (USDA, 2002). In addition to horse numbers, the Ameri- 
can Horse Council concluded that the equine industry has a dramatic 
impact on the United States economy. In terms of the Gross Domestic 
Product (GDP), this industry contributes a total of $39 billion, which 
is greater than the motion picture industry, railroad transportation, fur- 
niture manufacturing and tobacco product manufacturing services (The 
American Horse Council, 2005). In Kentucky alone, the equine industry 
has a reported economic impact of more than one billion dollars (Center 
for Business and Economic Research, 2004). For the first time, the 2010 
World Equestrian Games will be hosted in the United States in Lexing- 
ton, Kentucky. This event will create jobs and internship opportunities 
in several aspects of the industry. Recently, as a greater percentage of 
students from urbanized communities have enrolled in applied animal 
science programs (Britt et al., 2008; Hoover & Marshall, 1998; Reiling 
et al., 2003), development of standardized curriculum and assessment to 
meet the needs of students with either extensive or non-existent animal 
handling skills is necessitated. 

Although educational opportunities in the equine industry have in- 
creased over the last 20 years, there is no universally adopted skill set 
required for graduates of existing equine programs. Potential career 
paths require mastery of both hands-on skills and business knowledge 
(Conners & Brady, 2009; Houge-Davies, 2004; Kretler, 1995). For in- 
stance, an equine professional having direct contact with horses should 
be able to assess the health and welfare of the horse, sense and inter- 
pret physical changes with their hands and visual inspections, interpret 
behavior, perform training of the horse for a specific purpose, assess 
and manage feeding and nutritional needs, communicate with clients and 
medical professionals, and perform accounting tasks plus many more 
business management activities (Conners & Brady, 2009; Houge-Davies, 
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2004; Landers, 2002). Clearly, the wide array of skills necessitated for 
such careers confounds development of academic standards for teaching 
equine studies. 

The Equine Studies Program at Midway College consists of two 
Baccalaureate degrees and one Associate degree. The Baccalaureate de- 
grees are Bachelor of Science in Equine Studies with Concentration in 
Equine Health and Rehabilitation and Bachelor of Arts in Equine Studies 
with Concentration in Applied Equine Management. All degree pro- 
grams require mastery of hands-on skills in addition to specific cognitive 
and assessment tasks. At entry, the majority of students display some 
level of equine experience, but there is neither a common level of pro- 
ficiency nor a similar training background among students. To address 
these differences, a four course sequence of Practicum classes is required 
of all equine studies students at Midway College. 

Hands-on skill learning is often evaluated only by achievement of 
the assigned task after repetitive practices. Similar to rote memoriza- 
tion of facts, this repetitive performance can be perceived as boring or 
insignificant, thereby decreasing intrinsic motivation to learn a new skill 
(Char, 2009). Further, pre-existing knowledge of equine handling does 
not correlate to re-learning skills in a safe method (Meek et al., 2005). 
When students do not perform a skill following the in-house method, it 
is more difficult to reduce potential safety hazards. Besides safety, the 
learning process is often disrupted because inexperienced students are 
initially not as comfortable handling horses and look for guidance from 
experienced students (Meek et al., 2005). Applied learning with horses 
is unlike chemistry experiments where mixing of specific chemicals pro- 
vides a predictable reaction (or no reaction). Horses are unpredictable 
animals that react to the environment to satisfy basic survival needs. 
Therefore, this unpredictability poses a problem in standardizing the 
teaching and learning process and can present interference as less con- 
fident students express lower motivation in situations that are perceived 
to be unsafe. In consideration of these challenges, a teaching tool was 
investigated as a means to link the process of critical thinking to action. 

Generally, rubrics are part of a teacher’s daily routine in which ru- 
brics are created and utilized to accurately achieve a grading standard. 
Rubrics are a form of authenticity set to guide both students and teach- 
ers in assignment evaluation and are often subjective in nature. They 
are commonly seen in writing courses and used for assessment testing 
in primary schools (Andrade, 2000; Andrade & Du, 2005; Char, 2009; 
Loveland, 2005). According to Loveland (2005), rubrics are vital be- 
cause of the need for clear descriptions of project expectations, and upon 
review, rubrics provide a clearly delineated path for students to improve 
their work. Loveland also stated the benefits of a well-planned and thor- 
ough rubric for teachers. Development of an original rubric requires 
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instructors to take a retrospective view of a task for which they are writ- 
ing the rubric (Leonhardt, 2005). Other authors echoed the need for 
rubrics to be clear and concise (Andrade, 2005; Andrade & Du, 2005; 
Issaacson & Stacy, 2009). If students do not have prior knowledge of 
the categories defined on a rubric, then rubric usage can create frustra- 
tion, be dismissed, or be used only in a partial format (Andrade & Du, 
2005). To increase effective use of rubrics, students should practice us- 
ing the assessment tool (Andrade, 2000; Hafner & Hafner, 2003; Tan & 
Towndrow, 2009). Often students will use a rubric as a study guide or 
outline for developing written projects (Andrade & Du, 2005), medical 
task performance (Brown et al., 2006), and formative development of 
music skills (Leonhardt, 2005) indicating that such an assessment tool is 
useful in the teaching process. 

For this research project, a rubric was chosen as a teaching and as- 
sessment tool because each hands-on task, or psychomotor skill, could 
be subdivided into parts and performance measures could be evaluated; 
thereby the process of teaching and learning the task would be standard- 
ized. In theory, providing the assessment tool prior to a task should im- 
prove student scores; therefore, the objective of Phase I was to determine 
if prior knowledge of the assessment tool would increase overall scores 
as well as scores on individual steps as compared to groups that did not 
receive the rubric. Based on results from Phase I, a second objective was 
developed to determine if student motivation to perform hands-on skills 
is enhanced when provided with an affective rubric for self-assessment. 


METHOD 
PHASE I 


A faculty-derived rubric was developed for a showmanship pattern, 
which is a skill commonly found in showing horses for competition and 
sale. This task is universal in that all breeds and disciplines in the in- 
dustry use showmanship for marketing their horses and/or service busi- 
nesses. Although some variations in horse presentation and handler dress 
exist, efforts were made to prepare a rubric which would be applicable 
across breeds and disciplines. The showmanship skill involves the stu- 
dent making preparations with a horse several weeks prior to completing 
the task; therefore, students would require adequate time to prepare for 
assessment. Steps to create an original rubric were modified using a 
previously described method (Leonhardt, 2005). The rubric was subse- 
quently revised, based on student and faculty feedback following an ini- 
tial review as previously suggested (Murthy & Etkina, 2005). First-year 
students (n = 55) were either randomly provided (PR) or not provided 
(NP) the rubric three weeks prior to skills testing. In this instance, the 
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task was desegregated into 11 categories with a pass superior, pass, or 
fail scoring category worth 2, 1, and 0 points, respectively (Table 1). 
Once the students were assessed using the rubric, data were analyzed 
by the Proc GLM procedure of SAS with judge and whether or not the 
rubric was provided as test variables. Significances were determined at 
p < 0.05 with tendencies noted between p = 0.05 and p = 0.10. 


PHASE IT— PILOT STUDY 


As the equine industry incorporates non-invasive performance en- 
hancing techniques such as hydrotherapy, massage, and water treadmill 
exercise, there is increased demand for individuals with both knowledge 
and the ability to synthesize and evaluate healing and physical condition 
visually and through tactile sensations. Unlike human rehabilitation, 
horses do not provide reliable feedback on how the student is applying 
touch; furthermore, educating students to have ‘thinking hands’ presents 
a challenge. A holistic approach to teaching sensory perception was 
implemented in a series of manual integration courses. The activities 
were designed in a sequential format to promote movement through all 
six cognitive levels (Bloom, 1956), and using methods from an experien- 
tial learning model previously applied in a large-animal practical course 
(Reiling et al., 2003). Tools for teaching touch in other fields, such as 
nursing, were used in conjunction with lecture materials. Further, labo- 
ratories were developed using heating packs and false hair samples to 
simulate heat and cold perception. During a touch laboratory students 
were expected to apply self-massage techniques on the arms, hands, 
shoulders, and head to explore feeling. Students were also assigned to 
apply massage techniques on a project horse for the course of two semes- 
ters. Students were required to self-assess affective characteristics of 
“comfort” and “feeling” (Table 2) immediately following completion of 
aforementioned exercises. Students were then asked to reflect on the ex- 
perience in a written format and were led in an informal group discussion 
regarding the affective rubric to determine usefulness and motivation to 
learn the novel tasks. 


RESULTS 
PHASE I 


Sample tasks and assessment standards included a measurable out- 
come for many of the showmanship steps (Table 1). For example, the 
task of pivoting a horse 90 degrees is measurable by the amount of pivot. 
However, some tasks were considered more subjective by the raters than 
those with a clearly quantifiable value. For each task category, students 
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earned a score (Fail = 0, Pass = 1, Pass Superior = 2) that was compiled 
to equate to a total score. An overall score of 11 was considered passing 
for the assessment. The results demonstrated that the PR students earned 
a lower (p < 0.05) total score of 12.8 than the NP students earned of 17.1 
(Table 3). In consideration of individual score categories (Table 4), the 
PR and NP students had similar (p > 0.05) pass superior scores. Interest- 
ingly, PR students demonstrated higher (p < 0.05) pass scores than the 
NP students. Conversely, higher (p < 0.05) fail scores (1.4) were noted 
in NP students as compared to PR students (0.1). 


Table 2: Assessment rubric for comfort and feeling 


sore in my hands 
and shoulders; I 


drained nor 
energized; I 


drained nor 
energized; I feel 


Comfort I was/am 20% I was/am 40% _ | I was/am 70% I was/am 100% 
comfortable with | comfortable comfortable with | comfortable with 
performing this with performing this performing this 
exercise. performing this | exercise. exercise. 

exercise. 

Personal Tam exhausted;I | Iam neutral;I | I am neutral; I Iam energized or 

Feeling feel drained; I feel | feel neither feel neither at least feel the 


same as I started 
the session; I feel 


am 100% feel sore in my | good overall; I grounded; I feel 
dissatisfied with hands and feel sore in my only minor 
the session shoulders; Iam | hands and soreness in my 
only 25 % shoulders; I am hands and 
satisfied with only 70% shoulders; I am 
the session satisfied with the | 100% satisfied 
session with the session. 


Table 3: Total scores for students (n = 55) either provided the rubric (PR) or not provided the 
rubric (NP) prior to the skill assessment. 


Student Group Total Score! 
PR 12.8° 
NP 17.1° 
(5.3) 


‘Total scores for all rubric categories (mean square error for total scores) 
*» Values with unlike superscripts within a column are significantly different at p < 0.05 


Table 4: Individual score categories for students (n = 55) either provided the rubric (PR) 
or not provided the rubric (NP) prior to the skill assessment. 


Score Category’ 
Student Group Pass Superior Pass Fail 
PR 2.3" 6.1° 0.1° 
NP 3.3° 2.0° 1.4 
(3.5) (2.9) (0.2) 


‘Individual scores for all rubric categories (mean square error for total scores) 
*» Values with unlike superscripts within a column are significantly different at p < 0.05 
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PHASE II 


A standardized rubric (Table 2) for assessing affective values in per- 
forming tactile skills was rated as 100% (7 of 7 responses) useful by stu- 
dents. When asked to analyze individual progress over the semester, stu- 
dents indicated that their comfort and personal feeling scores decreased 
under two conditions: when a task was novel or when external stressors 
(exams, personal situations, lack of rest) impacted their participation. 
Faculty observations of student involvement in course activities noted 
that as students realized that their peers were also feeling uncomfort- 
able or tired, the class was more likely to participate in discussions and 
expressed more eagerness to apply the techniques in a practicum setting. 


DISCUSSION 


Results from the initial study agree with the concepts identified in 
both the arts and music fields (Mason & Steedly, 2006; Meier et al., 
2006; Leonhardt, 2005). There were notable discrepancies in the use of 
individual steps of the rubric, making assessment complicated. However, 
based on focus group discussions with raters, the faculty-derived rubric 
was desegregated into appropriate categories and considered useful, al- 
though measurement of ‘usefulness’ was not determined in Phase I.From 
these discussions, the importance of providing a method for quantifying 
performance under each task category was noted. 

Although not clearly addressed, other works utilizing rubrics for psy- 
chomotor skills often include some form of quantitative measure (Brown 
et al., 2006; Seybert & Barton, 2007), although such values were not 
consistently included in rubrics used for writing skills (Andrade, 2005; 
Loveland, 2005). By nature, the rubric as an assessment tool should 
make grading less redundant by limiting the number of times a professor 
writes the same comment and provides a means for student understand- 
ing of grading standards (Brookhart, 2003). Descriptors such as “few,” 
“wide variety,” “slowly,” “many,” are open to interpretation resulting 
in differences between rater and student interpretation. The differences 
between the need for quantitative measure in rubrics for cognitive skills 
as compared to psychomotor skills may be related more to field of study. 
For instance, in nursing, a “likely harmful” score in a rubric dealing with 
application of aseptic technique for preparation of aseptic products may 
result in injury, illness or death depending on the ultimate use of the 
product (Brown et al., 2006). In consideration of an increased risk to 
humans or animals, rubric categories should be quantifiable by time, 
proportions, or other forms of measure to limit rater interpretation and 
subjectivity. As a means to address clarity in rubric use, proportions 
were implemented into the rubric developed for Phase II. The use of 
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proportions gave raters the ability to assess the value of their affective 
response on a commonly known scale. These alterations aided in creat- 
ing clarity of the assessment tool, which is in agreement with previous 
work (Andrade & Du, 2005; Meier et al., 2006). 

Murthy and Etkina (2005) designed a rubric to assess their students’ 
abilities in devising solutions to a laboratory problem and conveying 
their solution in a written format reports in a large enrollment introducto- 
ry laboratory course. The average final exam score for the sample group 
was 78.3 as compared to the class average of 75 out of 100 possible 
points. The authors indicated a need for development and revision of the 
rubric prior to use and suggested that student performance improved af- 
ter receiving the rubric (Murthy & Etkina, 2005). These data contradict 
the findings in Phase I. Apparently students in Phase I PR group were 
motivated to accomplish a “passing” score, but not motivated enough to 
achieve a “pass superior” score. Further, there were more students to 
both “fail” and “pass superior’ in the NP groups reflecting the highest 
and lowest motivation for achieving the overall skill. 

Compilation of student reflections into several themes demonstrated 
in both phases of this study that the overall experience of using a rubric 
was positive and alleviated much of the confusion often associated with 
the “how-to’s” of hands-on skills. The students in Phase II reflected 
that the rubrics should be created for all equine courses to help alleviate 
subjectivity in assessment. This theme supports the faculty observation 
that perception of fairness in assessment and clearness of expectations 
enhances motivation and willingness to learn these skills and agrees with 
similar findings in the writing field (Andrade & Du, 2005) and for oral 
presentations (Hafner & Hafner, 2003). Despite this positive theme, 
there were student-derived recommendations for improvement of the as- 
sessment tools. The first recommendation theme was that some steps in 
the skills rubric should be weighted as critical points, and the skill steps 
should be even more detailed to make the teaching and assessment tool 
more effective. Another recommendation was that all students could 
have benefited from having the tool in advance for practicing hands-on 
skills; however, based on student scores in this trial, this perception does 
not result in achievement of the highest possible score. This discrepancy 
between attitude towards the rubric use and perception of performance 
level was also observed in the application of music skills (Char, 2009; 
Schmidt, 2005). Assessment of a student’s motivation to practice a task 
as compared to performance of that task may be a better means to dif- 
ferentiate intrinsic from extrinsic motivators (Schmidt, 2005). 
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USE OF RUBRICS AND STUDENT DEVELOPMENT 
IN APPLIED LEARNING 


Bloom’s Taxonomy of Learning dictates that learning occurs in three 
different manners: cognitive (knowledge), affective (attitude), and psy- 
chomotor (skills) (Bloom, 1956). In an equine studies program, a base 
level of cognitive learning is necessary so that the student understands 
theory and background common to the industry. Also necessary is a 
high level of psychomotor learning in an effort to achieve practical ap- 
plications. This presents a challenge to the instructor who must balance 
the levels and application of cognitive and psychomotor learning within 
a given lesson. In addition to complicating the teaching process, the 
combination of cognitive and psychomotor elements creates a challenge 
in assessing student performance. Students demonstrating a particular 
task (evidence of the skill) are not usually expected to also demonstrate 
cognitive learning (either in verbal or written format). Conversely, stu- 
dents demonstrating cognitive learning generally are not required to 
demonstrate psychomotor learning with a horse in the classroom envi- 
ronment. 

A framework describing the different stages of psychomotor learn- 
ing has been previously described (Simpson, 1972). Within this frame- 
work a student progresses through the steps of perception, set, guided 
response, mechanism, complex overt response, adaptation, and origina- 
tion. These divisions progress from basic learning of a skill (percep- 
tion and set) to advanced mastery of the skill resulting in spontaneous 
improvement as a response to specific problems (origination). Within an 
equine studies program, professors encourage students to progress from 
guided response to at least the mechanism step. As students progress to 
the mechanism category, they have moved into the intermediate stages 
of learning. During this type of psychomotor learning, response and 
actions become habitual, and the student displays a moderate amount of 
confidence when displaying skills (Simpson, 1972). Students in equine 
programs are traditionally assessed by demonstration and subsequent 
comparison of the psychomotor skill to a standard, which is defined in- 
house or by a respected equine professional. However, achievement of 
the skill is not always the entire purpose of teaching the task, and some 
methods of performing individual skills put both the horse and student 
in potentially dangerous situations. These skills are often not intuitive 
or inherently logical; therefore, safety issues and program inconsisten- 
cies dictate the need for these skills and the learning experience to be 
standardized. In the equine industry, judging standards associated with 
showing of various breeds and disciplines exist; however, clear holistic 
standards that reach across all breeds or disciplines do not. In other 
fields, there are standardized tests for outcome-based cognitive learning. 
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The data presented in this trial indicate a combination of an assessment 
rubric developed for a specific psychomotor task and an affective rubric 
for self-assessment are viable options for teaching and assessing applied 
skills in the equine studies field. 

Previous experiences with handling horses affect student comfort in 
performing applied tasks (Meek et al., 2005). Through further develop- 
ment of rubrics for other equine studies skills, as indicated in this study, 
the hands-on experience could be standardized. Furthermore, well-con- 
structed rubrics may also assist in developing intrinsic motivation for 
learning these skills when students demonstrate pre-existing horse expe- 
riences. Inexperienced students could benefit from these tools by having 
a clearly delineated step-wise process. Both Andrade and Du (2005) and 
Brookhart (2003) reported that formative assessment of “good writing” 
results in students transferring their rubric-based conceptions of quality 
work into other courses and to other students. As academic programs 
in the equine studies field grow, further development of teaching and 
assessment tools which standardize hands-on skill learning and the criti- 
cal thinking process would promote cohesiveness of equine studies pro- 
grams, enhance student motivation, and provide a means for document- 
ing student progress. Other fields of study that do not have the benefit 
of multiple leading generations might also benefit from development of 
similar tools to promote standardization, motivation, and documentation 
of student learning. 

Experiential learning and more specifically active learning is an in- 
structional method which engages students in the learning process and 
requires that students think about what they are doing instead of solely 
memorizing facts (Lohuis et al., 1999; Prince, 2004; Tan & Towndrow, 
2009). In this study, rubrics were used as applications of universal intel- 
lectual standards (Paul & Elder, 2001) for teaching and assessment of 
hands-on skills. Based on focus group discussions, the student learn- 
ing experience was positive, and the tools served as effective means for 
linking critical thought to action. Students were encouraged not only to 
perform a skill, but to evaluate how to perform that skill. This observa- 
tion has also been recently documented in students learning music skills 
(Char, 2009). By using rubrics in this method, the process of critical 
thinking was introduced in a non-threatening manner. Students clearly 
struggled with converting from cognitive to affective and self-reflective 
activities as indicated through observable expressions of frustration and 
dissatisfaction in Phase II. These observations support the data from 
Phase I of students being more goal-oriented than process-oriented. The 
results from Phase I clearly delineated that first-year students may not 
work to the highest score level when provided with the assessment tool, 
which is in agreement with results in the music (Char, 2009) and writing 
(Andrade & Du, 2005) fields. Motivation in third- and fourth-year stu- 
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dents is often perceived as less about comfort and more about focusing 
on graduation or employment goals. Clayton (2009) stressed the de- 
velopment of learning objectives for “critical reflection [that] generates, 
deepens and documents learning” (p. 7). Similarly, in nursing clinical 
education, the use of rubrics aided not only in instructor feedback, but 
also student application of critical thinking and reflection (Isaacson & 
Stacy, 2009). In making the shift from Phase I to Phase II, course ac- 
tivities were aligned with moving students through a clearly delineated, 
step-wise learning experience. Although a small data set (n = 7), student 
feedback in Phase II supports the need for critical reflection previously 
outlined (Andrade, 2000; Clayton, 2009; Isaccson & Stacy, 2009), not 
only for documentation, but also to enhance motivation and engagement 
in the learning process. 

One student’s reflection regarding the use of rubrics in Phase II 
stated, “...creativity in teaching starts with the ability to teach in unex- 
pected or unlikely situations. Teaching is not a matter of the quantity 
of the content, but the quality of learning. Although at first I disliked 
the tasks and found them cumbersome, as the semester progressed, 
these tools helped me assess my quality of learning in a way that had 
not been previously given to me. I find myself now thinking—what 
proportion of the task did I really complete? Am I comfortable and pre- 
pared? What did I learn?” This reflection exemplifies the concept of an 
engaged pedagogy and a shift in student perspective. Both engagement 
and perspective shifts are goals of applied learning and critical reflec- 
tion (Clayton, 2009). As further evidence of this shift in perspective, 
anonymous sharing of student responses fostered discussion and greater 
willingness of student participation as observed by increased discussion 
in the classroom and peer-to-peer interactions in Phase II. Students rated 
their comfort level greater when the task was associated with the knowl- 
edge or comprehension levels than when the activity required analysis, 
synthesis, or evaluation. These reflections also raised interesting data 
regarding specific tactile sensations. The most difficult sensory percep- 
tion to teach is temperature sensitivity. This perception is confounded 
when the ambient temperature is near freezing, such as that found in an 
unheated stable during the winter months. Development of standardized 
models aided in teaching this sense, although more work on developing 
reliable models is necessitated. Application of affective rubrics in other 
fields may aid in further elucidating the issue of student motivation. 


FUTURE DIRECTIONS AND CONCLUSIONS 
As an unexpected result, when techniques were demonstrated stu- 


dents focused more on trying to mimic exact movements, instead of 
reflecting on their own perceptions. When techniques were not dem- 
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onstrated, students developed their own approach to the task. The au- 
thors are currently investigating demonstration of a skill as compared 
to written or verbal explanation of skills, in relation to the learning and 
assessment process. This research direction builds on previous research 
(Meek et al., 2005) in combination with the questions regarding assess- 
ment found in this study. As a model for distance learning (either on- 
line or video conferencing), previously unpublished data regarding stu- 
dent perception of PowerPoint presentations was investigated in a small 
group of students (n = 14). For the test course, the content was presented 
solely through verbal explanation and writing on a white board. All stu- 
dents had previously been in equine studies classes which were taught 
using primarily PowerPoint lectures. Two questions were asked on a 
voluntary survey to determine student preference of delivery method: 1) 
I preferred that the professor wrote on the dry-erase board for teaching 
the course content; 2) I would have preferred that the professor had pre- 
pared all PowerPoint presentations for teaching the course content. Stu- 
dents evaluated these questions using a five-point scale from “strongly 
agree” to “strongly disagree”. Student preference of dry-erase board was 
100% “strongly agree” or “agree”. Interestingly, only a total of 33% of 
students “agree” with the use of PowerPoint presentations for teaching 
content, without any “strongly agree” answers. The next step was to 
discern why students preferred the use of the dry-erase board. During 
an informal discussion, a main theme emerged. Students preferred the 
hand-written material because the course pace was slower and discus- 
sion more interactive than when using PowerPoint or other technology. 
This feedback conflicts with use of digital video technology in biology 
as an assessment of student-teacher interaction (Tan & Towndrow, 2009) 
and use of music recording software to motivate beginner musicians 
(Char, 2009). In equine studies, there is an increased pressure to be 
competitive with online programs such as business or health care admin- 
istration. The experiential learning aspect of equine studies and other 
agricultural fields is what draws students to these programs (Lohuis et 
al., 1999; Meek & Marean, 2006; Reiling et al., 2003). In consideration 
of student feedback regarding PowerPoint utilization in the traditional 
classroom and Phase II observations presented, delivery method of ap- 
plied skills and subsequent cognition of “how-to” perform that skill are 
not equitable in terms of learning and attitude towards learning. There- 
fore, the value of assessment tools, such as skills rubrics, combined with 
method of delivery warrants further investigation in an applied learning 
model. 

Although the overall research objective was to promote standardiza- 
tion of hands-on skills, these data raise several interesting questions re- 
garding applied learning pedagogy. Rubrics aided in student motivation 
to learn and perform skills when provided early in the learning process; 
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however, students may only work to a passing level or other level which 
they feel is acceptable for their personal goals. The assessment rubric 
assisted students in de-mystifying a novel task in the equine studies field, 
as documented in the focus group discussions. Tasks should be aligned 
with well-defined objectives which provide purposeful progression, and 
have measureable outcomes. Demonstration as compared to description 
of tasks delineated on an assessment rubric warrants more research. Stu- 
dents expressed motivation for and engagement in the learning process 
when using affective rubrics for self-assessment. 
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Changes in the demographic composition of the United States relative to the increase in 
English language learners (ELLs) in newcomer and refugee populations generate the need 
for cultural brokers, particularly in the ranks of educators and teacher candidates. Applied 
learning, especially as used in service-learning opportunities in teacher candidate prepa- 
ration programs, can produce educators with greater cultural understanding and skills in 
best practices for literacy instruction. Examples of service-learning for this purpose are 
highlighted, as well as emerging strategies in professional development for educators who 
work with ELLs and their families. 


“America is on its way to becoming a microcosm of the entire world. 
One out of every ten people is foreign-born. One out of every five school 
children is foreign-born or had foreign-born parents. We are literally 
becoming a country made up of every country in the world.” So said 
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Kenneth Prewitt, former director of the United States Census Bureau in 
2000 (Pipher, 2002, p. 55). By the midpoint of this century, our nation 
will be even more racially and ethnically diverse, according to projec- 
tions released in August 2008 by the Bureau. Today’s minorities cur- 
rently comprise about one-third of the U.S. population and are expected 
to become the majority by 2042; the nation is projected to be 54 percent 
of former minorities by 2050. In 2023, today’s minorities will comprise 
more than half of the U.S. population of children as well. By 2050, chil- 
dren are expected to be 62 percent of former minorities, compared to 44 
percent today; thirty-nine percent are projected to be Hispanic, up from 
22 percent in 2008. The Hispanic population is projected to nearly triple 
during 2008-2050, and its percentage of the nation’s total population is 
projected to double, from 15 percent to 30 percent—meaning nearly one 
in three U.S. residents would be Hispanic (Bernstein & Edwards, 2008). 

As a country made up of many countries throughout the world, the 
United States is facing a demographic tipping point—a cultural learn- 
ing curve not yet seen in its history. When newcomers and members of 
the mainstream culture speak different languages, the learning curve is 
especially steep on both sides. The U.S. Census Bureau reported that in 
2000, 4.4 million households encompassing 11.9 million people were 
linguistically isolated—nearly double that of the previous decade. Lin- 
guistic isolation is defined by the Bureau as a home in which no one aged 
14 or over speaks English at least “very well” (Shin & Bruno, 2003). Al- 
though mainstreamers may assume that newcomers do not want to learn 
English, many speak multiple languages already and are indeed learn- 
ing English (Pipher, 2002), even though it takes most English language 
learners (ELLs) from one to three years to learn social English and five 
to seven years to learn academic English (Sutton, 1998). 

The U.S. Department of Education defines ELLs as national-origin- 
minority students who have limited English proficiency. ELLs represent 
one of the fastest growing groups among the school-aged population in 
this nation as well, increasing by over 169% from the years 1979 to 
2003 (Francis, Rivera, Lesaux, Kieffer, & Rivera, 2006). Of those, the 
number who spoke English with difficulty (i.e., less than “very well”) 
grew by 124% (Hill & Flynn, 2006). Over 400 different languages are 
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spoken in this group; Spanish is most common, spoken by 70% of ELLs 
(Francis et al., 2006). In fact, ELLs are expected to comprise 30% of 
school-aged population by 2015 (Hill & Flynn, 2006). This dramatic 
increase in the number of ELLs attending U.S. schools over the past 25 
years has revealed critical learning, language, and acculturation needs of 
students and families. Who is best positioned in our culture to address 
these needs? How can it be done? 


EDUCATORS AS CULTURAL BROKERS 


Cultural brokering is defined as the act of bridging, linking, or me- 
diating between groups or persons of differing cultural backgrounds for 
the purpose of reducing conflict or producing change (Jezewski, 1990, 
as cited in National Center for Cultural Competence, 2009). A cultural 
broker acts as “a go-between, one who advocates on behalf of another 
individual or group” (National Center for Cultural Competence, 2009, p. 
1). Cultural brokers can help to ease people into each other’s cultures by 
assisting newcomers and members of the mainstream culture to navigate 
the tensions of finding balance between respect for ethnic traditions and 
respect for human rights (Pipher, 2002). Cultural brokers also can help 
ELLs acquire language through meaningful use and interaction in differ- 
ent social settings and for different purposes (Campbell, 2010). Where 
is the best place to find such cultural brokers? In our schools. 

Schools are frontline institutions for acculturation, where children 
receive the information they need about the world in which they now 
live; therefore, educators are the most important cultural brokers in our 
society. Nearly all newcomer and refugee families have tremendous re- 
spect for education and educators (Pipher, 2002). They recognize the 
need to learn the language, and hence the culture, as soon as possible; 
children want to learn English, learn in school, and fit into their new so- 
ciety (Campbell, 2010). Because language learning is cultural learning, 
every language reflects the norms, behaviors, and beliefs of a unique cul- 
ture; therefore, “the learning of a new language also involves the learning 
of new norms, behaviors, and beliefs” (Campbell, 2010, p. 316). Pipher 
(2002) also states, “To really become American, refugees must become 
both bilingual and bicultural” (p. 76). Furthermore, teaching language 
requires educators to learn the culture of the language: “All language 
learning is cultural learning” (Heath, 1986, as cited in Campbell, 2010, 
p. 315). 

To be effective cultural brokers, educators must be able to commu- 
nicate with students and their families. When educators facilitate lan- 
guage learning, they serve as cultural brokers and help students and their 
families to work through cultural conflict and encourage them to be- 
come empowered to have control over their destiny. This facilitation of 
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language learning occurs most appropriately and effectively by building 
on what students and families already know. The concepts, language, 
and strategies that children have already acquired at home, their “funds 
of knowledge,” are the connecting points upon which to teach new con- 
cepts, language, and strategies (Campbell, 2010). Rejection of children’s 
home language and culture may result in responses that include failure 
to learn, withdrawal and passivity, and/or power struggles with and re- 
sistance to teachers and the school culture (Campbell, 2010). Students’ 
native language and culture may be treated as inferior, not worthy of 
being used as tools in the school setting. “In fact, when we do not allow 
students to build on their existing language, we and they lose a great 
deal of richness and value, and we perpetuate the myth that non-English 
languages are not school languages” (Campbell, 2010, p. 320). 

The challenge, then, lies in developing educators who are much bet- 
ter prepared than those in previous generations to teach in linguistically 
and culturally diverse classrooms and to effectively use the funds of 
knowledge that children and their families bring to the school experi- 
ence. Of course, classroom teachers or even ELL teachers cannot learn 
all there is to know about the various cultures in schools. Educators can, 
however, better interpret the cultures of diverse student populations in 
their schools through awareness of how much culture affects language 
acquisition and behavior, insight into their own culture, and discernment 
not to interpret the behavior of others through the eyes of their own cul- 
ture (Haynes, 2005). This awareness, insight, and discernment can cer- 
tainly be developed through professional development with experienced 
teachers, but initially—and perhaps most importantly—it can be devel- 
oped through the applied learning experiences of teacher candidates and 
educators. 


CULTURAL BROKERING AS SERVICE-LEARNING 


Applied learning emphasizes the relevance of what is being learned 
to the world outside the classroom, and makes that connection as im- 
mediate and transparent as possible so that students can focus on learn- 
ing and applying the skills and knowledge they need to solve a prob- 
lem, implement a project, or participate in the work force (Victorian 
Curriculum and Assessment Authority, 2006). This results in the for- 
mation of partnerships and connections with individuals and organiza- 
tions outside school that provide the necessary out-of-school contexts for 
students to demonstrate the relevance of their learning. Research indi- 
cates that experiential immersion is the best way for students to develop 
empathy, tolerance, respect, and appreciation for people, languages, 
and cultures different from their own (Michie, 2003). Noddings (2005) 
concurs that “social action can and must ‘extend beyond schools walls 
to the wider world’ where faculty create learning opportunities at an 
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appropriate developmental level for students that help raise their aware- 
ness of social justice by allowing them to experience, to be with the 
‘other,’ for whom we must care” (p. 7). 

When teacher education programs structure applied learning expe- 
riences that allow teacher candidates to interact with people different 
from themselves, teacher candidates are empowered to construct their 
knowledge base about diversity from life experience. The assumption is 
that these teacher candidates, with a richer background of experiences 
in culturally diverse settings, will be better prepared to teach culturally 
diverse students than those teacher candidates who have lived insulated, 
monocultural lifestyles and who have had limited experiences incul- 
turally diverse settings. Modest, and sometimes dramatic, changes can 
occur in teachers’ attitudes and behaviors regarding diversity (Smith, 
1998). Although the role of educating people to be cultural brokers is 
hardly touched on in the scholarly literature (Michie, 2003), a review of 
empirical studies concluded that experiences with members of diverse 
populations are worthwhile for teachers, and positive results can accrue 
for students and teachers having the contexts and support within which 
to interpret their experiences (Smith, 1998). Applied learning can pro- 
vide both the context and the support for both teacher candidates and 
educators to become cultural brokers. 

These applied learning experiences, when structured as service- 
learning, can help teacher candidates learn about other languages and 
cultures, to have greater involvement in international affairs, to put their 
own culture in perspective and see it in a new way, and to simply feel 
useful and helpful. Pipher (2002) states, “Having a cultural broker can 
make a tremendous difference in how successfully a new family adapts 
to America. People come here traumatized, and the trauma doesn’t end 
with arrival. Without guidance and support, it’s difficult to survive” (p. 
85). Pipher’s suggestions for cultural brokering lend themselves well to 
service-learning applications in teacher preparation courses. She advo- 
cates visiting with newcomers and refugees whenever possible, on the 
streets or in the stores, and especially when seeing someone looking lost 
or confused. Helping when and where a need is identified is at the heart 
of service-learning, such as tutoring in after-school programs or for GED 
and ELL classes and volunteering at agencies that serve immigrants and 
refugees. Following is one such service-learning opportunity that took 
place in central Nebraska as the result of a coordinated effort between 
the University of Nebraska at Kearney (UNK) and Educational Service 
Unit (ESU) 10 in Kearney, Nebraska. 


FLAME: SERVICE-LEARNERS AS CULTURAL BROKERS 


The UNK College of Education requires teacher candidates to com- 
plete a service-learning experience as a prerequisite for admission into 
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the teacher education program. Students who enroll in the entry-level 
course in the professional education course sequence are required to 
complete their service-learning experience during the semester they take 
the course; logistics for student placements are coordinated by the direc- 
tor of the Office for Service-Learning in collaboration with the faculty of 
the course (University of Nebraska at Kearney, 2009). 

Teacher candidates were first given the opportunity to put cultural 
brokering skills to use in a service-learning setting in a program initiated 
by ESU 10, the local educational resource agency. ESU 10 is a regional 
education agency that provides supplementary educational services to 
K-12 school districts in eleven counties covering nearly 12,000 square 
miles in central Nebraska, serving about 30,000 students and approxi- 
mately 2,200 teachers (Educational Service Unit 10, 2009). In 1999 the 
ELL program of ESU 10 secured a multiyear Title VII Development 
and Implementation Grant from the U.S. Department of Education to 
develop, adapt, and implement the Project FLAME (Family Literacy: 
Aprendiendo, Mejorando, Educando) program with eight community 
sites, involving 663 parents and 910 children 2% through 12 years of 
age, 14 teacher education candidates, 26 teachers, 18 paraprofession- 
als, and 12 school administrators throughout the three year duration of 
the program. Specific cultural and literacy skills training was provided 
by the grant to 136 educators within the target communities (ESU 10, 
2002). FLAME was the program of choice by the ESU 10 ELL program 
after an extensive search was conducted on family literacy programs; 
those with medical models (“something is wrong with you”), military 
models (“war against illiteracy”), or patronizing tones (“if you were just 
more like us, you’d be better’) were rejected. The “La Familia” philoso- 
phy of FLAME respects Hispanic culture; family is the most important 
social unit, so the program involves the entire family. FLAME utilizes 
an asset model that builds on the strengths that families and students al- 
ready bring: their language and their culture, the connecting points upon 
which new language and cultural knowledge are built. The FLAME proj- 
ect in central Nebraska has helped families provide circumstances and 
interactions that support literacy skills, whether in Spanish or English, 
using culturally-relevant practice and all conducted with the tone of re- 
spect. 

Project FLAME was first designed by Professors Flora Rodriguez- 
Brown and Timothy Shanahan of the University of Illinois at Chicago 
(UIC) College of Education in 1989 as part of a request for funding from 
the U.S. Department of Education. The original purpose of the program 
was to support parents of preschoolers and primary grade students by 
providing information and sharing knowledge about ways to provide a 
home environment rich in literacy learning opportunities for their chil- 
dren. FLAME is based on the belief, supported by research, that parents 
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can positively affect their children’s learning and school achievement by 
providing a supportive home environment and serving as confident, suc- 
cessful learner models. Additionally, FLAME believes that successful 
family literacy is sensitive to the social, linguistic, and cultural contexts 
of the families served. The FLAME curriculum has twelve family litera- 
cy sessions that include topics covering book sharing, book selection, us- 
ing the library, and creating home literacy centers; helping children learn 
the alphabet, read, write, do homework, and learn songs and games; and 
helping parents connect to the community through classroom observa- 
tions, field trips, and collaboration with other parents. The curriculum 
is not rigidly structured and is intended to be responsive to participants’ 
needs and concerns. Adoption sites are encouraged to revise, expand, 
and adapt each lesson to parental needs and community context (UIC 
College of Education, 2003). 

In central Nebraska, the curriculum was adapted to put additional 
emphasis on increasing access to books and literacy materials. Parents 
were encouraged to attend ESL and GED classes already offered in their 
own communities, and literacy activities for children replaced baby- 
sitting. Instead of using graduate-level students as in the original UIC 
implementation, ESU 10 collaborated with the UNK College of Educa- 
tion to provide service-learning placements in the FLAME program for 
teacher candidates who assisted with literacy activities for the children 
and their parents. Service-learners participated in meet-and-greet with 
the families upon their arrival; children would go to a separate area to 
work with UNK students who read books to them and followed up with 
literacy activities such as helping the children write their own stories in 
a particular pattern. At the same time parents would meet with FLAME- 
trained educators from the eight community sites in sessions that cov- 
ered literacy topics mentioned previously. After the literacy activities 
were completed in both groups, parents were shown what their children 
had experienced, and then children shared with their parents what they 
had done. The children read to their parents, and UNK students modeled 
for parents how to interact with their children appropriately regarding 
the readings and engaged with family members regarding their chil- 
dren’s work. This was followed by social time with food and raffles for 
literacy materials that were conducted by the children themselves so they 
had practice reading numbers. Most of the UNK students did not speak 
Spanish; they had to figure out how to communicate with ELLs and learn 
not to be afraid to speak to someone different from themselves. Most 
students also sang and read in Spanish even if they were not Spanish 
speakers (ESU 10, 1999). 
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STUDENT IMPACT 


A key outcome for these service-learners was letting go of the fear 
of communicating. They moved from hanging back with their fellow 
college students, to being given a job to do and doing it. They were 
able to develop close relationships with the children; older Hispanic chil- 
dren who spoke English for the most part were able to talk to the UNK 
students and received encouragement to go to college. Service-learners 
were surprised at how much they enjoyed participating with the families 
while developing a deeper understanding of La Familia. Excerpts from 
their service-learning logs follow (ESU 10, 2002): 


“One of the most important things that I learned during my ser- 
vice-learning experience was respect for others, no matter how dif- 
ferent they are from me. The groups of children that I worked with 
were all Hispanic and many were from very low-class families. Be- 
fore this experience I would not have looked at these children and 
their families with the same empathy I do now...I have learned that I 
cannot judge people based on appearances.” 

“T interacted with them and could actually tell they understood me 
even though I was speaking English or not speaking at all because 
people have an unspoken language that everyone can understand. 
However, I did start to use some of my Spanish that I learned in high 
school and thought that I had long time forgotten [sic]. They were 
simple words and phrases but I think it shows that I cared ...I was at 
first a bit hesitant about working with all these people that spoke a 
different language than me but now I realize that as I continue with 
my field experience I can make an impact on them and they will 
definitely make an impact on me.” 

“When teaching I need to use more than just my words to teach . . 
. Lam now excited for any opportunity that may arise for me to work 
with other languages besides English.” 

“T love this program. I think it is a fabulous idea to get the parents 
involved with their children’s education and to help build confidence 
in the children . . . when I become a teacher I will have a variety of 
children and cultures and learning about other cultures helps me to 
understand my students and what method of teaching works best for 
them.” 

“After being placed with this literacy program for a semester, I was 
able to put forth my Spanish speaking skills into real life situations. 
What I learned from both the children and parents was beyond what I 
anticipated. They taught me so much about their language and it was 
wonderful being able to share with them by means of communication.” 

“Trying to communicate in a language other than my native lan- 
guage is difficult and sometimes frustrating, but also very good for 
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me. I think it helps me to develop more of a cultural sensitivity and to 
understand the difficulties these families must have here in the United 
States.” 

“T am very thankful for having the opportunity to work with Project 
FLAME. As a result of FLAME, I have found myself to [be] a more 
open-minded and knowledgeable person. This program has also mo- 
tivated me to pursue an endorsement in the ESL [English as a Second 
Language] area. I have seen how effective this program has been and 
I hope that I, as a teacher, will be able to help my students as much as 
FLAME has helped theirs.” 


All of the students completing the service-learning experience, with- 
out exception, stated that they experienced a transformation from initial 
biases about minorities and language learning to greater understanding 
and acceptance. All students were from small, rural Nebraska towns, 
with limited exposure to and experience with minority populations of 
any kind prior to attending UNK. The consistent outcome for these stu- 
dents was an increased sense of confidence as they prepared to teach in 
classrooms much more diverse than their hometown schools had been 
for them. Only one student comment reflected a less positive sentiment, 
that of dislike for some of the necessary “office work” that went into pre- 
paring for the sessions. Yet, even this was a valuable learning experience 
as well: “I now realize how much work planning just one lesson is. . . I 
will definitely be organized when it comes to me planning and creating 
my own lessons” (ESU 10, 2002). 

Because the service-learning component was not specifically includ- 
ed in the original objectives of the FLAME grant as submitted to the U.S. 
Department of Education, program evaluations from the UNK service- 
learners were not a part of the grant reporting process. Therefore, assess- 
ment of the service-learning component in future project implementation 
would certainly benefit from more structured pre-session orientation and 
post-session debriefing, in addition to the reflective journaling for the 
UNK service-learning course and the informal preparation and debrief- 
ing with ESU 10 staff that took place on the way to and from sessions. 


CLIENT IMPACT 


Evaluation results of the original piloting of FLAME in Illinois indi- 
cate that children of families who participated showed significant gains 
in cognitive development, pre-literacy and literacy skills, and vocabu- 
lary development in both Spanish and English. Results further showed 
that parents changed their attitudes towards teaching their children and 
also became more proficient in English, evidenced by significant gains 
in English proficiency as measured by the Language Assessment Scales 
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(Rodriguez-Brown, Shanahan, & Wagner, 1999). These claims were val- 
idated by the Offices of Bilingual Education, Community Affairs, and 
Early Childhood of the Illinois State Board of Education, which allowed 
FLAME to qualify for a dissemination grant from the U.S. Department 
of Education in order to carry the program nationally as a family literacy 
adoption model. In 1996, the U.S. Department of Education awarded 
Project FLAME a five-year Academic Excellence Grant to export the 
program to train over fifty adoption sites, including the one in central 
Nebraska (Zygouris-Coe & Smith, 2001). 

In Nebraska, the ESU 10 FLAME program mirrored these results; 
80% of parents who attended FLAME classes increased the number of 
home literacy activities with their children and improved their children’s 
access to reading and writing materials as evidenced by self-reports. On 
an evaluation scale asking parents to rate the degree to which the pro- 
gram accomplished its objectives, the composite score across all sites 
was ninety out of 100 points (ESU 10, 2002). Although the parents pre- 
viously felt they were not able to help their children because they could 
not speak English, now they realized they could help them in Spanish 
through the transfer of Spanish skills to English in vocabulary, content, 
and concept development. Additionally, parents participated more in 
school activities, attended school events more often, and served as rep- 
resentatives of other parents (Bransford, 2002). Following are excerpts 
from parent evaluations of the ESU 10 FLAME program (ESU 10, 2002) 
[translated from Spanish to English]: 


“It is nice to know that there is someone interested in the Hispanic 
community, and especially who supports and is interested in our chil- 
dren.” 

“T like the way that you support our children’s education and also the 
way that you have the children working in school activities and the 
way that teachers treat us when we are together, and the talks that you 
give us on how to help our children to read better.” 

“T learn very many different things so that I can help my daughters 
with their homework.” 

“It is a very important help for our children’s education. Thank you.” 


“Me gustaria que este programa continuara mas anos” — “T would 
like this program to continue for more years” was the sentiment consis- 
tently expressed by parents who completed program evaluations in the 
final year, and it was the only concern they raised. In several instances, 
parents also expressed a desire to continue sessions during the sum- 
mer and for longer hours “para aprender mas para ayudar a nuestros 
hijos” —“‘so that we can learn more and help our children.” Participating 
families reported a stronger sense of connection to the communities that 
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were now their homes and expressed a desire to help to secure future 
program funding so that their children could continue to benefit. Without 
exception, their remarks focused on helping their children first (ESU 10, 
2002). 

Additionally, the Nebraska FLAME program provided scholarships 
for teacher candidates, paraprofessionals, and educators to pursue ESL 
teaching endorsements; sixty-nine scholarships were awarded during the 
3-year project period. The program was recognized as an outstanding 
educational program by the Foundation for Educational Funding and the 
Nebraska Association of Student Financial Aid Administrators in April 
2002 (ESU 10, 2002). 


POST-FLAME IMPACT 


Though the Project FLAME grant concluded in 2002, it served as a 
catalyst for UNK teacher candidates to seek cultural brokering oppor- 
tunities with ELLs in subsequent years, and for UNK teacher education 
faculty to use those opportunities as service-learning experiences for the 
teacher education program. Project FLAME utilized about 10% of the 
total number of service-learners from the UNK teacher education pro- 
gram, which averages approximately 150 students per semester. Now the 
percentage of service-learners participating in cultural brokering-related 
activity with ELLs, including language and literacy development, has 
grown to nearly half. In central Nebraska communities where the His- 
panic ELL population is significant (20% in Grand Island, Nebraska, 
for example), and in the same communities where Project FLAME had 
been conducted, cultural brokering opportunities in service-learning for 
teacher candidates has taken place in public school outreach centers 
and cultural welcome centers. Activities have included serving meals 
for the Salvation Army; sorting, stocking, and distributing food pantry 
items; participating in coat drives; reading to children and participating 
in poverty simulations during National Homeless Month; serving needy 
families through the Christmas drives during the month of December; 
packing backpacks on Friday mornings for weekend food for children 
in need; teaching and assisting with citizenship classes; facilitating a 
moms’ group in learning life skills; providing help and advice for fami- 
lies regarding applying to higher education; assisting with the 2010 cen- 
sus and encouraging citizens to vote. 

Unfortunately, the number of teacher candidates seeking an actual 
ESL endorsement plummeted to zero by 2003; the waning of scholarship 
monies at the end of the ESU 10 grant was a contributing factor. But ac- 
cording to Dr. Glenn Tracy, lead instructor of the ESL endorsement at 
UNK, the continuation and expansion of service-learning experiences 
for teacher candidates has helped the number rebound to a steady ten to 


120 Journal of Applied Learning in Higher Education / Fall 2009 


twelve in the last several years, and there were thirteen enrolled in the 
program at the end of the 2008-2009 school year (personal communi- 
cation, July 9, 2009). Of the sixty majors available to the nearly three 
hundred teacher candidates enrolled at UNK, the ESL endorsement now 
ranks as the fifth most chosen endorsement area (University of Nebraska 
at Kearney, 2008). 

In addition, educators have access to ongoing training in service- 
learning as pedagogy for their own K-12 and college classrooms in a 
graduate level course available from the UNK teacher education depart- 
ment and taught by the director of the Office of Service-Learning. Over 
sixty faculty members from multiple disciplines outside of the teacher 
education program have participated in training institutes to learn how to 
incorporate service-learning into their courses. In this way, the notion of 
cultural brokering with ELL populations is expanding beyond beginning 
level teacher education students into classrooms and schools throughout 
central Nebraska. 

Concurrent with the growth of cultural brokering service-learning 
experiences for teacher candidates, Nebraska’s own state Department 
of Education, in collaboration with the North Central Regional Com- 
prehensive Center and the Nebraska Council of School Administrators, 
embarked on a professional development plan for educators who were 
facing changes in classroom and school ELL demographics. In 2007 the 
Nebraska English Language Learner Leadership Institute (NELLLI) 
was initiated to increase teachers’ and administrators’ understanding of 
research-based instructional strategies with ELLs, to increase partici- 
pants’ understanding of the correlation between leadership practices and 
student achievement, and increase participants’ capacity to lead ELL 
reform efforts in their respective districts and schools. Representative 
teams of educators and administrators in 26 school districts across the 
state, as well as one team representing higher education teacher prepa- 
ration programs, applied to become part of multi-day quarterly train- 
ing sessions. The teams have taken ELL best practices for language 
acquisition and cultural brokering strategies back to their districts and 
provided leadership for ongoing professional development within their 
own schools. In addition, a design team was formed from the NELLLI 
training participants to provide ongoing support to teams, and eventually 
all schools in Nebraska (though web access will make universal access 
possible) through the development of an online resource center called 
the Continuous Improvement Process Toolkit, accessible through the 
Nebraska Department of Education website (Nebraska Department of 
Education, 2009). 

The first task of the NELLLI design team was the creation of a mis- 
sion and vision statement (NELLLI, 2009): 
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THE TEAM MISSION: 

We will equip all Nebraska schools to effectively meet the educational 
needs of English Language Learners by creating readily accessible re- 
sources through the Continuous Improvement Process Toolkit. 


THE TEAM VISION: 

All English Language Learners will receive research-based instruction 
in the English Language and content areas so that they are empowered 
to progress toward academic mastery and achieve their fullest potential. 
All students in Nebraska schools will respect language and cultural dif- 
ferences and communicate effectively with each other in English and 
conversationally in a second language. 

All PreK-16 educators will receive ongoing training in exemplary cur- 
riculum and instructional strategies that promote social and academic 
English development and cultural identity development. 


The remaining tasks include the completion of ELL training resourc- 
es that are user-friendly for educators and administrators in a variety 
of school contexts, and the completion of a book study to be used as a 
professional development tool in conjunction with the book Classroom 
Instruction That Works With English Language Learners, by Jane Hill 
and Kathleen Flynn (2006). The NELLLI design team seeks to complete 
their work on the toolkit by the end of 2009. 

Even more recently, the passage of The American Recovery and Re- 
investment Act (ARRA) of 2009 has provided new funding for address- 
ing the needs of English language learners: “Our recommendations are 
focused on ELLs because they represent a large proportion of students 
at risk of underachievement in states and districts across the country” 
(Stanford University, 2009). The funding guidance for this act states that 
parents of ELLs are insufficiently engaged in schools and in educational 
decision-making, so ARRA Title I funds can be used to improve avenues 
for engaging parents in their children’s education, perhaps resulting in a 
resurgence of programs such as Project FLAME. 


CONCLUSION 


Teacher candidates and educators are key cultural brokers within their 
respective communities, and service-learning opportunities provide a 
powerful pedagogy through which to develop their knowledge and skills. 
By embracing cultural diversity and language acquisition best practices, 
teacher candidates and educators can collaborate to support and empower 
ELLs and their families to connect with the resources they need to attain 
success in the mainstream culture and to effectively prepare for becom- 
ing the majority culture in the very near future. In doing so, they embrace 
our moral imperative, as global citizens, to care (Noddings, 2005). 
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